When going to Lifecycle Management – Bundle Management in SDDC Manager, I saw the following two error messages:
Retrieving all applicable bundles failed. Encountered fetching http://127.0.0.1/lcm/inventory/upgrades api - FAILED_TO_VALIDATE_SDDC_MANAGER_COMPATIBILITY; Failed to validate if SDDC Manager with version 5.0.0.1-22485660 is compatible with system.
Retrieving available bundles failed. Unable to retrieve aggregated domains upgrade status: Encountered fetching http://127.0.0.1/lcm/inventory/upgrades api - FAILED_TO_VALIDATE_SDDC_MANAGER_COMPATIBILITY; Failed to validate if SDDC Manager with version 5.0.0.1-22485660 is compatible with system.
Research led me to the following KB:
SDDC UI throws error “COMPATIBILITY_MATRIX_CONTENT_FILE_NOT_FOUND” in 5.x versions
Going to Administration – Online Depot shows VMware Customer Connect Depot as Active.
The KB told me to verify that required URLs are not blocked, per this KB:
Public URL list for SDDC Manager
Trying to connect to all the URLs in the KB from SDDC Manager succeeded:
curl -v https://depot.vmware.com
curl -v https://vcsa.vmware.com
curl -v https://vmw-vvs-esp-resources.s3.us-west-2.amazonaws.com
curl -v https://vvs.esp.vmware.com
curl -v https://vsanhealth.vmware.com
Then I found a way to download the VmwareCompatibilityData.json file manually to see if that would work:
root@vcf001 [ /home/vcf ]# grep vcf.compatibility.controllers.vvs.api.endpoint= /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties
vcf.compatibility.controllers.vvs.api.endpoint=vvs.esp.vmware.com
root@vcf001 [ /home/vcf ]# curl --location 'https://vvs.esp.vmware.com/v1/products/bundles/type/vcf-lcm-bundle?format=json' --header 'Content-type: application/json' --header 'X-Vmw-Esp-ClientId: vcf-lcm' > VmwareCompatibilityData.json
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:04:59 --:--:-- 0
curl: (28) Failed to connect to storage.googleapis.com port 443 after 298678 ms: Couldn't connect to server
As we can see I was trying to download from the allowed URL https://vvs.esp.vmware.com, but it fails to connect to storage.googleapis.com so it seems to redirect to a URL that is not listed in the KB. Allowing connections to storage.googleapis.com from SDDC Manager solved the issue and SDDC Manager was immediately able to download bundles again. I contacted Broadcom support who confirmed this and later updated the KB with the correct URL.