VCF 9 licenses now available for VMUG Advantage members
For a long time, access to personal-use VCF licenses have been available to anyone with a VMUG Advantage membership as long as they also pass a VCP-VCF certification. However, until recently these licenses were only available for VCF 5.x, but now we also get licenses for VCF 9. I have tested them in my lab…
Deploy an NSX Edge Cluster in VCF 9
For the vSphere Supervisor that I deployed in my last post to work I needed to deploy an NSX Edge Cluster. This can now be done in the vSphere Client so that is what I chose to do to get some experience with this process. This is done by going to the vCenter for the…
Deploy a Workload Domain with vSphere Supervisor in VCF 9
This post will show you how I deployed a new Workload Domain in VMware Cloud Foundation 9 (VCF 9) with the vSphere Supervisor enabled. vSphere Supervisor lets me provision and manage virtual machines, containers and full Kubernetes clusters through vSphere Kubernetes Service (VKS) on my VCF platform. Broadcom’s documentation has a nice summary of what…
Deploying VCF Operations for Logs in VCF 9
When deploying a new VCF Fleet, Operations for Logs is not deployed and needs to be done using VCF Operations later. As you can see from the image below, VCF Operations and VCF Automation (automation) are labeled with “New Deployment” since they are deployed, while VCF Operations for Logs (operations-logs), VCF Identity Broker (identity broker)…
Deploying VCF 9 using a JSON Specification File
I wanted to redeploy my VCF 9 lab and thought I would use the JSON spec file that I exported when deploying it using the wizard the first time. I started with logging in to the VCF Installer Appliance and selected Deploy Using JSON Spec instead of Deployment Wizard. After uploading my JSON spec file…
Deploying VMware Cloud Foundation (VCF) 9 in my Lab
VCF 9 was released this week with a lots of news you can read about here:https://blogs.vmware.com/cloud-foundation/2025/06/17/whats-new-in-vmware-cloud-foundation-9-0/ Since I work a lot with VCF I wanted to see if my lab was able to run the new version. My newest server has 512 GB RAM, 2 x Intel Xeon Gold 6138 processors and 2 x 2 TB NVMe.…
The effect of using MAC Learning in ESXi nested labs
When using nested ESXi we have to enable either Promiscuous mode or MAC Learning on the VDS on the physical host running the nested environment. Forged transmits is also required to be enabled. I changed to MAC Learning long ago since I knew Promiscuous mode had a performance impact. I have had great results running…
FAILED_TO_VALIDATE_SDDC_MANAGER_COMPATIBILITY error message in SDDC Manager
When going to Lifecycle Management – Bundle Management in SDDC Manager, I saw the following two error messages: Retrieving all applicable bundles failed. Encountered fetching http://127.0.0.1/lcm/inventory/upgrades api – FAILED_TO_VALIDATE_SDDC_MANAGER_COMPATIBILITY; Failed to validate if SDDC Manager with version 5.0.0.1-22485660 is compatible with system. Retrieving available bundles failed. Unable to retrieve aggregated domains upgrade status: Encountered fetching…
Upgrade to SDDC Manager 5.2.1.0 fails with “502 Bad Gateway”
A while after kicking off the SDDC Manager 5.2.1.0 upgrade I was presented with the nginx error “502 Bad Gateway” in my browser. Thinking that this was caused by the services being restarted as part of the upgrade I waited 15 more minutes but the error would not go away like it use to. Checking…
Failed uploading the update/upgrade patch files to VUM when upgrading to VCF 5.2
I did the precheck for a VCF 5.2 upgrade after uploading the HPE ProLiant custom iso for VMware ESXi 8.0 Update 3 Build 24022510 and was greeted with the following error: Bundle validation for bundle of type HOST with upgrade version 8.0.3-24022510 failed due to error Failed uploading the update/upgrade patch files to VUM,performing compliance…
“Cannot change immutable field: tier1_lr” error in VMware Cloud Director 10.5.1
We experienced that doing changes to a Virtual Service on a Load Balancer in the VMware Cloud Director (VCD) user interface gave us the following error message after upgrading to VCD 10.5.1: Cannot change immutable field: tier1_lr We were using NSX Advanced Load Balancer (ALB) version 22.1.5 Build 9093, but the issue seems to be…
VMware Cloud Foundation 5.0.0.1 to 5.1 Upgrade Notes
I recently upgraded a customer from VMware Cloud Foundation (VCF) 5.0.0.1 to 5.1. The upgrade went well in the end, but I had some issues along the way that I would like to share in this quick post. The first issue I ran into was that I was unable to select 5.1 as target version…
Upgrade VMware Cloud Director using Cloud Provider Lifecycle Manager
I wanted to test the NSX Advanced Load Balancer Self-service WAF which came with Cloud Director 10.5.1. My lab was running 10.5.0 so I needed to upgrade. First step was to download VMware_Cloud_Director_10.5.1.10593-22821417_update.tar.gz from VMware and copy it to /cplcmrepo/vcd/10.5.1/update on the Cloud Provider Lifecycle Manager appliance. Then I chose Manage and Upgrade on my…
Using Terraform to Commission Hosts and Creating Clusters in VCF
I don’t have much experience with Terraform but one of my customers use it a lot and they wanted me to make a proof of concept on how to use it to commission hosts and create a new cluster in VCF. This is a very simple first edition based on examples found here. After running…
VCF 4.5.1 to 5.0.0.1 Upgrade Notes
I recently upgraded a customer from VMware Cloud Foundation 4.5.1 to 5.0.0.1. The upgrade went well in the end, but I had some issues along the way that I would like to share in this quick post. Hopefully you won’t run into these issues at all, but if you do, perhaps this post can help…
Deploy VMware Cloud Director using Cloud Provider Lifecycle Manager
I have recently started working with a VMware Cloud Service Provider where I have the role as an architect. They are running VMware Cloud Director (VCD) on top of VMware Cloud Foundation (VCF), and while I have quite a bit of experience with VCF, I haven’t worked a lot with VCD so I thought I…
iperf3 on VMware ESXi 8 gives “Operation not permitted” error message
I have been using iperf3 to test network performance between ESXi hosts or between an ESXi host and another device running iperf3. Earlier we could simply copy the iperf3 file to a new file like iperf3.copy and run that but this no longer work in ESXi 8 due to security being hardened. Here is a…
Boot VMware ESXi from NVMe on servers not supporting NVMe boot
I have a Dell PowerEdge T440 which doesn’t support booting ESXi directly from my PCIe NVMe drive. This server does not have any other controllers or drives, so my only option is to boot it from USB/SD. Since booting ESXi from USB/SD is deprecated I found another solution. Clover is a bootloader with many features…
Extending My VCF Lab Across Multiple Physical Hosts
I have been running VMware Cloud Foundation (VCF) in a lab for a few years now. It has always been running on a single physical host. This is fine as long as the host is powerful enough but lately I have been thinking about how to extend it across more than one physical host. I…
Creating VLAN Backed Segments in VMware Cloud Foundation (VCF)
When deploying VCF the hosts are prepared for NSX using a Transport Node Profile (TNP) where only a single Overlay Transport Zone is attached and no VLAN Transport Zones. This means you are unable to create any VLAN backed segments. I have been asked by several customers how to use VLAN backed segments with VCF…
Deploying VMware Cloud Foundation (VCF) 5.0 in my Home Lab
I have an 11 years old Dell PowerEdge T420 server in my basement. This server already ran ESXi 7 and 8 without problems (!) but it lacked the performance to do everything I wanted it to do, like running VMware Cloud Foundation (VCF). I investigated options to upgrade the server, and since it aleady had…
Upgrading my Lab to vSphere 8
I have a small two node vSAN cluster running at home on a couple of Intel NUCs. This cluster has been running vSphere 7 since it was born, but now I figured it was time to upgrade it to vSphere 8. I started with upgrading the vCenter Server by mounting the iso on my laptop,…
Issues When Creating Custom ESXi ISO Image Using vSphere Lifecycle Manager
I use vSphere Lifecycle Manager to create custom ESXi iso images when upgrading or installing ESXi for customers but the process has some issues that I will address in this post. The first issue is that you are unable to export the file because the port number in the URL is wrong and you get…
ESXi Temporary Directory Exist Error on VCF Upgrade Precheck
I helped a customer upgrading from VCF 4.3 to 4.4 and encountered the following error message during the Upgrade Precheck: “ESXi Temporary Directory Exist” Expanding the error message gave me the following error description with several remediation suggestions: “Directory /locker on host does not support the upgrade for resource, Please check” None of the suggestions…
Convert vSphere Lifecycle Manager Mode in VCF
I recently had a customer who was unable to upgrade the Management Workload Domain in their VCF 4.3 deployment. After digging around for a while I found that the cluster was configured with vSphere Lifecycle Manager Images which is not compatible with the following features which they were using: – Stretched Cluster -…
DHCP is crucial for VMware Cloud Foundation (VCF)
When deploying VMware Cloud Foundation (VCF) one can choose between configuring NSX-T Host Overlay using a static IP Pool or DHCP. This is specified in the vcf-ems-deployment-parameter spreadsheet which is used by the Cloud Builder appliance to bring up the Management Workload Domain as you can see in the following images: If No is chosen,…
vCenter 7.0 requires LACP v2
When upgrading to vCenter 7.0 one of the prerequisites to check is the versions of the existing vSphere Distributed Switches (VDS) in the environment. They need to be at version 6.0 or higher. Recently I was helping a customer upgrading to vCenter 7.0 and all their VDS’es were at version 6.6.0, and all other prerequisites…
Unable to query vSphere health information. Check vSphere Client logs for details.
I have been helping quite a few customers with upgrading to vSphere 7.0 lately. In this post I will share an issue we encountered when upgrading from vCenter 6.7 to 7.0 and how we resolved it. Prior to starting the upgrade I always do a thorough health check of the existing environment. The following error…
Running vSAN Witness in The Cloud
vSAN Stretched Cluster requires three independent sites; two data sites and one witness site. If you don’t have a third site, you can run the vSAN Witness Appliance in the cloud. This post will show you how I deployed a vSAN Witness Appliance in Proact Hybrid Cloud (PHC) which runs VMware Cloud Director. I started…
New Undocumented Feature in NSX-T 3.2 MC
Migration Coordinator (MC) is VMware’s tool to migrate from NSX Data Center for vSphere (NSX-V) to NSX-T Data Center (NSX-T). Last week I was helping a customer migrating their Distributed Firewall Configuration from NSX-V to NSX-T 3.2 and we were presented with a new step in the process called “Prepare Infrastructure”. I have never seen…
Deploy Nested VCF to NSX-T Overlay
I have used VLC to deploy nested VCF for a long time and I am quite happy with how it works. VLC is usually deployed to a VLAN Trunk Port Group. This requires the VLANs used in the nested VCF to be configured on the physical switches in the environment. This does not scale well,…
Upgrading to VMware Cloud Foundation 4.4 in my Lab
VMware Cloud Foundation 4.4 was just released so I wanted to check out what was new and upgrade my lab. Going into SDDC Manager and selecting Lifecycle Management and Release Versions gave me an overview of what is new: Flexible vRealize Suite product upgrades: Starting with VMware Cloud Foundation 4.4 and vRealize Lifecycle Manager 8.6.2,…
NSX-v to NSX-T – End-to-End Migration #1
The End of General Support for VMware NSX Data Center for vSphere (NSX-v) was January 16, 2022. More details can be found here: https://kb.vmware.com/s/article/85706 If you are still running NSX-v, please start planning your migration to NSX-T as soon as possible as this may be a complex and time consuming job. I have done several…
Upgrading to VMware Cloud Foundation 4.3 in my Lab
VMware just released VMware Cloud Foundation (VCF) 4.3 and I have several customers planning to upgrade in the near future, so I decided to upgrade my lab to get some experience. I also have two customers planning to deploy VCF 4.3 on new hardware so I will also soon deploy it from scratch to see…
NSX-T Federation in my VMware Cloud Foundation (VCF) Lab
VCF 4.2 introduced support for NSX-T Federation which provides the ability to manage, control and synchronize multiple NSX-T deployments across different VCF instances which could be in a single region or deployed across regions. You can stretch Tier-0 and Tier-1 Gateways, Segments, and Groups used for Firewalling. Requirements between sites are maximum round-trip time of…
Multi-Instance Management in my VCF Lab
Multi-Instance Management (VCF Federation) allows you to monitor and manage multiple VMware Cloud Foundation instances from a single console. You can view inventory across the VMware Cloud Foundation instances in the federation as well as the available and used capacity. You can also see if any patches are ready to be installed in any of…
My first VMware PSO Project
Who is VMware Professional Services (PSO)? “VMware Professional Services, the largest services organization of experienced IT professionals focused solely on virtualization, provides industry-leading Consulting, Education, and Technical Account Manager services that enable customers to transform IT environments through virtualization and realize greater value from flexible, agile IT service delivery models sooner, with less risk.” I…
Avoid Packet Loss in NSX-T
I have been working a lot with NSX-T the last few years and I have come across a misconfiguration that may cause massive packet loss for the workloads connected to Overlay segments. Since NSX-T 2.5, the recommended Edge Node design has been the “Single N-VDS – Multi-TEP” design which looks like this: What people, and…
vCenter Server blocked by NSX firewall
Recently I had a customer calling me with panic in his voice. He had managed to create a rule in NSX where sources and destinations were both any, and action was set to drop. This rule was added high up in the rule set so almost all their workloads were blocked from the network, including…
VMware Cloud Foundation in a Lab
VMware Cloud Foundation (VCF) is basically a package containing vSphere, vSAN, NSX-T, and vRealize Suite elegantly managed by something called SDDC Manager. Everything is installed, configured and upgraded automatically without much user intervention. VCF is based on VMware Validated Design, so you get a well-designed, thoroughly tested and consistent deployment. Upgrading is also a lot…
Introduction to my Labs
Yes, I intentionally wrote Labs, as this post will introduce you to both my home lab and to the lab environment running in my employers data centers. I have just built a small home lab for the first time in many years. A lab is very important for someone like me who is testing new…
CyberNils.net 