Manage Learn to apply best practices and optimize your operations.

VMware load balancing gets tricky with large VMs

VMware increased the maximum VM size in vSphere 5.1. With large VMs comes a greater responsibility for VMware load balancing and availability.

Large virtual machines can test the limits of VMware load balancing and high availability technologies. VMware admins with large and important VMs -- like those running mission-critical apps -- need to keep performance high and resource utilization efficient to reap their benefits.

With the release of vSphere 5.1, VMware enabled larger virtual machines: 64 vCPUs and 1 TB of vRAM. Enterprises also are virtualizing complex, mission-critical apps, sometimes in multi-tiered vApps that use several VMs.

What do vSphere and vCloud admins have to watch out for when they deploy large VMs and multi-tiered vApps?

When deploying large VMs in vSphere or any other virtualized environment, pay close attention to load distribution and high availability balancing. Load distribution and high-availability balancing are not always easy to achieve, nor are automated vSphere load balancing and HA tools always up to the task in highly asymmetrical environments.

Absolute RAM and CPU consumption don't tell a VM's whole story. For example, a modern 2P server can easily host 512 GB of RAM. If a 384 GB VM resides on that server, with high RAM requirements but low CPU utilization, can you add a smaller VM with low RAM requirements but high CPU utilization?

Maybe, maybe not. There are many considerations. What percentage of the system's RAM bandwidth is consumed by that high-RAM large VM? What are the networking and storage I/O impacts on that host? There could be secondary effects on bandwidth caused by intense work bursts on RAM (or PCI Express peripherals). Admins will find that Distributed Resource Scheduler, the VMware load balancer, is not adept at digging this deep into resource usage.

Large, critical VMs face a considerable HA risk. If the primary host for a huge VM dies, what other host can take on the significant resource drain of this large VM? Do you even have a 384 GB block of RAM available on another host? The VM's other subsystems are likely to be affected by the VM's failover. Again, admins cannot always rely on the "balanced" readout built into the vSphere client.

Additional resources:


Dig Deeper on DRS and DPM

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Hi What about using an application level load balancer that can look at user connections and load balance to more small instances. Something like jetNEXUS is approved for use with VMware?
I agree, We try and use more smaller virtual machines and use an approved virtual load balancer to distribute the traffic. I do understand that this is not always possible especially with Database servers
Yeah...maybe the title should be Resource Load Balancing...just saying
Sure you can mix those together of big RAM with big CPU. Give the VM with high CPU requirements a lower priority but a decent reserve of CPU resources and give the VM with the lower CPU requirements but big RAM a higher CPU priority and a fair reservation over what it normally runs at. If you setup your reservations right than DRS and HA can do a decent job. If you don't and rely on ballooning, then yeah it may be an issue...