List Connected vCenter Servers with PowerCLI

List Connected vCenter Servers with PowerCLI

This functionality is pretty slick and is done automatically when connecting to vCenter with PowerCLI. It’s an oldie but goodie and I wanted to toss this out there to help anyone.

When you use Connect-VIServer to connect to a system, two specific global variables are updated.

Either of these can be used to display the last vCenter server you connected to. Even if you’ve made multiple connections, the last one will be listed.

List Multiple Connected vCenters with PowerCLI

A slight variation to this is to make these plural:

If you have made connections to multiple vCenter servers, those systems will be displayed here! These are extremely helpful when working to simplify things. You can even pass session information by simply doing one of the following:

vCenter SessionID

You can pass this information into a Job (with Start-Job) and connect to the vCenter servers you’ve already established a connection to. 

Read the rest →

Store Secure Credentials in the Registry with PowerShell and then connect to vCenter with PowerCLI

Store Secure Credentials in the Registry with PowerShell

This process will store secure credentials in the registry. It allows you to query those credentials and then pass them using PowerCLI to make a connection to vCenter.

This is a simple code snippet that will ask for credentials and then add them to the registry. To make this easier, you can have PowerShell create the key that will house the information and then add the proper values. I’m going to use HKEY_CURRENT_USER but feel free to store these in HKEY_LOCAL_MACHINE if necessary. To store information into HKLM, you may need to elevate your PowerShell environment to administrator. I find putting stuff in HKCU is much easier… and it has the added benefit of storing credentials with the user instead of the machine. 

This will add something like the following to your registry: 

Registry Secure Credentials Information

Notice that the Data column for administrator@vsphere.local will be the password you entered but in encrypted form.

Retrieve Secure Credentials from the Registry with PowerShell

Now that we have it stored, we need to pull the information and log into vCenter with PowerCLI.

Read the rest →

Upgrade vCenter with an ISO – Upgrade vCenter HA

Getting vCenter upgraded is fairly straight forward. However, if you have a vCenter HA configuration, it becomes a little more complex. Not difficult but there are some things you have to find before you can upgrade. I love VMware, don’t get me wrong. But, the documentation for upgrading vCenter HA is pretty…. vague and disjointed. So I wanted to go into detail to get through so that you know what to expect when upgrading.

This upgrade process is going to go over how to do this in a vCenter HA configuration. However, you can follow the same steps to do a non-vCenter HA deployment. You just won’t do any of the HA specific steps (skipping the upgrade on the passive and witness nodes). You’ll only update your primary vCenter instance and not the passive or witness nodes.

I’ve spent quite a bit of time with vCenter HA and have run into issues. I’ll mention those related to updating in this post. I do feel vCenter HA isn’t fully baked. It’s extremely quirky. 

vCenter HA Issues

  • You cannot upgrade a vCenter HA cluster unless it’s in maintenance mode. Even if you disable the vCenter HA cluster, it still won’t let you upgrade. Here is the message when I tried to updated the witness node when I’ve disabled vCenter HA:

vCenter HA installing patch is allowed only on passive or witness node

  • So, we put the cluster into maintenance mode before we upgrade. No worries right? Wrong. While you can update the witness and passive nodes, you cannot update the active node (it will throw a similar error as the one above). Well, why don’t I just failover the cluster? So if the passive and witness nodes are in this weird state (see pic below), you cannot failover. It’s like this chicken and egg situation:

vCenter HA status

  • You must acquire the vCenter patch ISO. You have to drill down into the docs to find this out… and frankly, it would be super confusing for someone that is just starting out. 

Read the rest →

Dell IDSDM not showing as a boot device Dell R640

Firstly I like these servers. They brought back the control panel/LCD on the front which is REALLY helpful when you are on the phone to someone remote. And the hardware you can run on them is amazing (2×28 core CPUs, 32TB RAM, NVDIMM support, M.2 support etc etc). 

However, one box I built for vSAN, caught me out.

No mater how may complete power downs, shutdowns, reboots and firmware upgrades I couldn’t get the Internal Dual SD Module (IDSDM) to show.

As you can see, no IDSDM
But it lists on Integrated Devices
However, from the DRAC it was there and happy. Although I couldn’t update its firmware.
I spoke to Dell to make sure I wasn’t going mental. They asked me to run through the ESXi install to see if ESXi could see it. And then, as if by magic, boom it worked. I could see the IDSDM along side my virtual optical drive and the other goodies you get from iDRAC

It appears it just needed a nudge and virtual media from iDRAC did it.
Hopefully this helps someone else who likes to configure their BIOS/ boot devices before they install.

This article and was done in coordination with Chris McChesney and Virtually Trivial.

Read the rest →

Reservations of VMs per host

Reservations are the devil.

The more you deep dive into reservations and resource pools the more you don’t want to touch them.

I wont go into great detail about the dangers as vFrank, Eric Sloof and Duncan Epping (to name but a few) have been telling us for years.

Basically unless you REALLY know what you are doing and you have a REALLY good reason to use them… Don’t.

However, back in the real world reservations are still being demanded by some vendors. Most notably anything to do with VOIP. Which is understandable as you don’t want these guys fighting for resources. I will say though some of the numbers they ask for can be eye watering.

One problem with reservations is that it can be quite difficult how much non reserved CPU MHz is actually left on a single host. vCenter will show you for the cluster but it doesn’t dive down to each host.

So you can have a scenario where you have a host with 10 VMs. 5 of them have reservations for half the CPU between them and the rest have none.

When the reservation VMs are doing nothing the CPU scheduler will allow the other VMs to use those cycles.

But.. what happens when the host gets busy. This is when things get tricky. If the reservation VMs are using 50% of the CPU then the other 5 hosts have to fight for the rest. If the reservations set are not high enough, the reservation VMs will ask for additional resources as well. Then you have to factor in hypervisor overhead for things like VSAN, iSCSI etc the list goes on.

Using DRS for VOIP things can be a no no or the host itself may not even be licensed for it.

So, what can you do to see how much actual free cycles are on each host?

Read the rest →

Updating ESXi Advanced Settings with Menu Driven PowerCLI

This code will create menus for you to update Advanced Settings (or edit it to do other things… it’s the menus that are most important) for your ESXi hosts. It connects to vCenter, creates a menu of clusters to select from, creates a menu of hosts to select from and then executes the change. It puts the host in maintenance mode, updates the value and reboots the host.

The code runs past the edge of the page, but there are controls on the below widget to expand it so you can see all of it. 🙂

Read the rest →

vCenter 6.7 Update 3 HA with DDNS – Disable DDNS on vCenter

As it stands today (11/11/2019), vCenter 6.7U3 has been released with a new DDNS “feature” that seems to be creating an issue when leveraging vCenter HA. The issue is that when vCenter updates DNS, it sends both the management IP and the HA IP. The HA IP should really never be seen by the end user. Since both IPs are updated in DNS via DDNS, you get a round-robin situation for the name that you have allocated to your vCenter appliance.

This will take you through updating vCenter so it doesn’t do DDNS and hence does not create two DNS entries for your vCenter server. Having said that, you will have to create a static DNS entry for your vCenter server (which is the normal operation when deploying vCenter anyway).

Please see the Release Notes to read more on the DDNS feature.

Disable DDNS on vCenter

  1. We are going to need to SSH into your vCenter server. 
  2. Type the following to remove the cron job. Feel free to back the file up.

    cd /etc/cron.d/

    rm -f dns_update.cron

  3. We now have to deal with a python script that runs. Type the following:

    ps -eaf | grep ddns

  4. This will give you a PID for the following:



  5. For this example, we are grabbing the 4431 PID and executing the following:

    kill -9 4431

  6. Let’s validate that we have killed it.

    ps -eaf | grep ddns

  7. Just make sure the ddns.py script isn’t running.
  8. Now we need to do an edit.

    cd /usr/lib/applmgmt/support/scripts/

    vi prestart-applmgmt.sh

  9. Since you are in vi, lets hit the / key and type: DDNS
  10. You will find the following lines that you need to comment out. Please see the below snippet of what needs to be commented:



  11. Note that we’ve added a # in front of each of the lines we no longer want to run.

Read the rest →

Deploying vSphere 6.5 Update 2 Platform Services Controller (PSC) and vCenter in an HA configuration [Part 4]

This will walk you through doing the vCenter HA deployment with the Advanced option. The reason you would use the Advanced option is if the vCenter VMs are not in the inventory of the vCenter that is going to be deployed as HA. This would mean that you deployed the VMs in another vCenter. There is good reason to deploy in another vCenter. Perhaps you have a vCenter already running and you will be migrating to this newer vCenter. The other reason might be that you deployed in a different vCenter, it will allow you to snapshot the VMs during your deployment.

This document will assume that you have deployed vCenter that will end up with an HA configuration into an already existing vCenter. This will facility a migration to the new vCenter once you’ve completed the HA deployment. If you have not deployed vCenter yet you can go here to do that: Deploying vSphere 6.5 Update 2 Platform Services Controller (PSC) and vCenter in an HA configuration [Part 3]

A supported vCenter HA configuration needs to have three separate hosts. One for each vCenter (two hosts) and one for the witness server (one host). If we did the Basic install, it would put the VMs on different hosts and add anti-affinity rules. You can follow this article about deploying with less than three hosts and the configuration change that is needed. Less than three hosts is not supported and should only be used in a test/lab environment.

Deploying vSphere 6.5 vCenter HA

  1. The first thing we need to do is edit the vCenter VM you have deployed already and add a NIC to it.
  2. Place the NIC on the VLAN/portgroup that will house the vCenter HA network (this is the non-public address space for vCenter HA… meaning, the communication done on this VLAN for vCenter HA is for the exchange of HA information.

Read the rest →

Deploying vSphere 6.5 Update 2 Platform Services Controller (PSC) and vCenter in an HA configuration [Part 3]

This is Part 3 of this process. If you need to review Part 2, you can do that here: Deploying vSphere 6.5 Update 2 Platform Services Controller (PSC) and vCenter in an HA configuration [Part 2]

This will walk you through deploying vCenter for HA. The deployment is actually very similar to deploying a non-HA configuration. This will use the same installer.exe that you use to deploy the PSCs.

Deploying vCenter 6.5 Update 2

Must haves

  • You have the Load Balancers setup for connecting to your PSCs
  • You have the PSCs setup properly with certs from the VMCA (this includes the VIP FQDN)
  • You have the DNS names created for any vCenter you will deploy (this includes the reverse DNS)
  • If you are able to take snaps of your PSCs while they are down (this would mean you are deploying into a different vCenter that enables snapshots to be taken). If this is a greenfield, you can take a snapshot of the PSCs while running, but I do not believe this is supported.

Procedure

  1. Launch the installer.exe
  2. Put a check in I accept the terms of the license agreement and then click Next
  3. Select vCenter Server (Requires External Platform Services Controller) and then click Next
  4. Type in the vCenter (or host) FQDN name that you will be deploying to and use the an administrator account (or root account if you are deploying to a host) to authenticate.
  5. Click Yes to accept the thumbprint
  6. Select the Datacenter you will deploy to and then click Next
  7. Select a host to deploy vCenter to and then click Next
  8. Type in the name of the appliance (this is the name that will show up in vCenter or on the host within it’s inventory) and the root password you want to use. Click Next
  9. Select the size you would like to use and then click Next
  10. Select the datastore and then click Next
  11. Set the following:
  • VLAN
  • System Name (this is the fqdn of the vCenter you are deploying)
  • IP Address
  • Subnet Mask
  • Default Gateway
  • DNS Server (you can enter just one for the time being)
  1. Click Next
  2. Validate that all the information entered is correct and then click Finish
  3. The deployment will start.

Read the rest →

Deploying vSphere 6.5 Update 2 Platform Services Controller (PSC) and vCenter in an HA configuration [Part 2]

This is Part 2 of this process. If you need to review Part 1, you can do that here: Deploying vSphere 6.5 Update 2 Platform Services Controller (PSC) and vCenter in an HA configuration [Part 1]

I want to point out a really good article: https://haveyoutriedreinstalling.com/psc-ha-6-5-1-introduction/psc-ha-6-5-2-prepare-a-load-balancer/psc-ha-6-5-3-preparing-a-certificate/

You may also want to make your VMCA a Subordinate CA. You can follow this article to accomplish that. Just repeat the steps for each PSC: https://www.virtuallytrivial.com/index.php/2018/10/05/vmca-6-5-update-2-as-a-subordinate-ca/

Yes, during the deployment of this configuration, I had to open a Support Request due to what I can only refer to as “bugs”. The case was escalated and the second tier engineer actually pointed me to that post. It is done by one of their VMware Engineers. Sadly, I had been referring to the article already. What I’m going to try to do is provide detail on Update 2 as there seems to be slight updates there.

Adjusting the Machine SSL Certificate on your Platform Services Controllers (PSCs) for Load Balancing

  1. SSH into your first PSC
  2. Type: cd /certs  (if the directory doesn’t exist, type mkdir /certs)
  3. Type: vi psc_ha_csr_cfg.cfg
  4. Type: i
  5. We need to paste in the following:

[ req ]
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req
[ v3_req ]
basicConstraints = CA:false
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = DNS:psc01.contoso.ad, DNS:psc02.contoso.ad, DNS:pscvip01.contoso.ad
[ req_distinguished_name ]
countryName = US
stateOrProvinceName = Washington
localityName = Seattle
0.organizationName = Contoso
organizationalUnitName = Contoso, Inc.
commonName = pscvip01.contoso.ad

Things to note:

  • subjectAltName = This needs to have the FQDN of all of your PSCs in that single site along with the FQDN of your VIP
  • commonName = This will be the FQDN of your VIP
  • Make sure you change countryName, stateOrProvinceName, localityName, 0.orgianizationName and organizationUnitName
  1. Once all of that is pasted in and edited hit ESC and then type: :wq
  2. Now type: openssl req -new -nodes -out /certs/psc_ha_vip.csr

Read the rest →