Large virtual machines can test the limits of VMware load balancing and high availability technologies. VMware admins with large and important VMs -- like those running mission-critical apps -- need to keep performance high and resource utilization efficient to reap their benefits.
With the release of vSphere 5.1, VMware enabled larger virtual machines: 64 vCPUs and 1 TB of vRAM. Enterprises also are virtualizing complex, mission-critical apps, sometimes in multi-tiered vApps that use several VMs.
What do vSphere and vCloud admins have to watch out for when they deploy large VMs and multi-tiered vApps?
When deploying large VMs in vSphere or any other virtualized environment, pay close attention to load distribution and high availability balancing. Load distribution and high-availability balancing are not always easy to achieve, nor are automated vSphere load balancing and HA tools always up to the task in highly asymmetrical environments.
Absolute RAM and CPU consumption don't tell a VM's whole story. For example, a modern 2P server can easily host 512 GB of RAM. If a 384 GB VM resides on that server, with high RAM requirements but low CPU utilization, can you add a smaller VM with low RAM requirements but high CPU utilization?
Maybe, maybe not. There are many considerations. What percentage of the system's RAM bandwidth is consumed by that high-RAM large VM? What are the networking and storage I/O impacts on that host? There could be secondary effects on bandwidth caused by intense work bursts on RAM (or PCI Express peripherals). Admins will find that Distributed Resource Scheduler, the VMware load balancer, is not adept at digging this deep into resource usage.
Large, critical VMs face a considerable HA risk. If the primary host for a huge VM dies, what other host can take on the significant resource drain of this large VM? Do you even have a 384 GB block of RAM available on another host? The VM's other subsystems are likely to be affected by the VM's failover. Again, admins cannot always rely on the "balanced" readout built into the vSphere client.
- VMware's Duncan Epping and Frank Denneman cover VMware load balancing in their book, VMware vSphere 5.1 Clustering Deepdive.
- Epping's and Denneman's work is reflected in Cloudphysics' offering, which is a simple HA monitoring solution designed for tricky VM load-balancing problems.
This was first published in March 2013