Improving vSphere performance can be as easy as removing an unnecessary virtual machine restriction, or as complex...
as restructuring queues to storage logical unit numbers.
VMware's Niran Even Chen, senior systems engineer and performance specialist, shared several vSphere performance best practices during his Boston VMUG 2013 presentation. These tips are a good place to start when vSphere performance doesn't match up with expectations, or when you're looking for the most bang for your buck in upgrades.
1. Avoid setting affinity rules. VMware vSphere will choose the best course for virtual machine assignments to available servers. Affinity rules are best relegated to use for testing and troubleshooting server problems.
2. Don't set resource limits. Resource limits, as opposed to resource reservations or shares, will prevent a VM from accessing more resources even if it needs them and the resources are free. Even if you haven't intentionally set resource limits, a bug in template cloning could have put some in place. If performance is artificially low, check for resource limits and remove them.
3. Update your hardware. Modern hardware is virtualization-aware and vice versa. For example, with Intel processors, you can leave Intel's Hyper Threading simultaneous multithreading chip technology turned on. VMware vSphere can use Hyper Threading to schedule more processes in the same amount of time.
4. Learn about the CPU Scheduler in vSphere. With relaxed co-scheduling, CPU Scheduler plans only some of the VM's CPU, not all of it, at once. This has a significant effect on latency compared to earlier CPU scheduling technologies.
5. Reduce VMs. Even idle vCPUs are using some amount of a physical CPU's capacity, wasting CPU cycles. Use vCenter Operations Manager (vCOps) reports and esxtop in real time together by investigating the READY and CO-STOP values for the CPU in vCOps. If the vCPU is waiting to use the physical CPU, the READY time will be off. If the application doesn't know how to consume all of its allocated virtual resources, the CO-STOP value will show this.
6. Don't over-allocate active memory. This causes a lot of swapping to disk, which is the very last memory management technique that vSphere wants to use.
7. Size applications within a non-uniform memory access (NUMA) node whenever possible. Some applications are NUMA-aware; others are not. For example, SQL automatically has NUMA recognition on, but Oracle has it off by default. Therefore, it's more important with Oracle to size the VM within the NUMA node. VMware vNUMA lets the guest OS see the NUMA architecture and typically boosts CPU performance 5% to 10%, and up to 30% in certain conditions. The memory management feature automatically turns on above eight vCPUs.
8. Use VMXNET3, the high-performance virtual network interface card (NIC), to save CPU power and reduce network latency for high network load VMs. This is a best practice that you should apply broadly.
9. Turn off interrupt coalescing on your hosts' NICs if the VM workload is network-sensitive. Interrupt coalescing, also called interrupt moderation, is meant to protect the host CPU from too many interrupts and streamline packet processing. But in certain conditions, it can increase latency, leaving the host waiting to be notified of new packets.
10. Check for latency in applications; application latency should be 0. If network latency is not causing a performance issue, check that the latency isn't coming from the application by looking into the packet transmissions.
11. Consolidate the queues to storage LUNs in collaboration with your storage administrators. Performance can suffer from multiple VMs trying to send I/O to the hypervisor at the same time, creating an I/O blender effect.