VM web console disconnected on vCloud Director 10.2

This is my first article on vCloud Director, It was an interesting day figuring out why the tenant’s VM web consoles were going from connecting to a disconnected state.

I am using vCloud Director 10.2. If you note, my public FQDN ssl certs are still valid, I exported my cert from the browser and replaced it with my public address cert from vCD provider -> Administrator -> Public Addresses. Let’s see how.

Step 1 : Exporting the Certificate

Click on the lock icon next to URL and view site information -> Connection is secure

It should say, “Certificate is valid” click on it. Go to details and export the full chain

Make sure you export the certificate chain, next we will replace this on VCD public addresses. (Thanks to my friend Ahmed for this)

Step 2 : Importing the cert to Public Addresses

Login to VCD provider and go to Administrator -> scroll down to Public Addresses -> Edit and replace it with the one we just exported.

Once you have replaced it, you should be able to see the rotated certs. Best part is once you click replace and choose the cert file it will populate the cert information and won’t take effect until you hit SAVE

After hitting save you should be able to see your Web Portal and API certs renewed

We have now renewed the Public Addresses Certificates, next comes the cell certificates which are for HTTPS and console proxy communication of VCloud Director this consists the embedded PostgreSQL database and the appliance management UI.

Note: I am using self signed certificates

You can also see similar errors on the console-proxy.logs from the cells


2022-09-12 13:06:39,979 | ERROR | consoleproxy queue : 0 | EncryptionTicketThumbprintAugmentation | Tampered ticket thumbprint C5:DA:10:20:36:80:C4:11:F6:0D:35:27:29:EE:2C:2A:00:C0:41:3C | java.nio.channels.SocketChannel[connected local=/10.11.110.101:8443 remote=/10.11.110.100:12490]
2022-09-12 13:06:39,979 | WARN | consoleproxy queue : 0 | WebsocketRequestHandler | Error processing websocket ticket: com.vmware.consoleproxy.ticket.AugmentationException: Tampered ticket thumbprint | java.nio.channels.SocketChannel[connected local=/10.11.110.101:8443 remote=/10.11.110.100:12490]
com.vmware.consoleproxy.ticket.AugmentationException: Tampered ticket thumbprint

Step 3: Rotating the vCloud director certificates

It is a good practice to take a DB backup and snapshot of the cells before renewing the certs. You can delete them after the certs are rotated.

Note: You will have to generate and import the certs on all the cells

SSH into one of the cell as root user.

We will first stop the VCD service, this won’t impact any of your running workloads

/opt/vmware/vcloud-director/bin/cell-management-tool -u administrator cell --shutdown

Given the condition of the cell, It might sometime now allow you to stop the service. Reboot the cell and re-run the above command to stop the VCD service

/opt/vmware/appliance/bin/generate-certificates.sh 'root-password'

Please give your root password inside ‘ ‘ This will generate a new self-signed certificates for HTTPS and console proxy communication of VMware Cloud Director plus certificates for the embedded PostgreSQL database and the appliance management UI.

As we are using self signed cert, let’s import the newly generated certificates to VCD

/opt/vmware/vcloud-director/bin/cell-management-tool certificates -j -p --keystore /opt/vmware/vcloud-director/certificates.ks --keystore-password 'root-password'

All that is left now is to restart the vCD service.