There are a few misunderstood resource controls in vSphere. CPU affinity is one of these. Controlling the delivery...
of CPU resources to a VM is an important part of performance management. CPU affinity is seldom a wise thing to configure on a VM and can be hard to remove once it is set. A VM with CPU affinity set will never get more CPU time than one without this setting.
No guarantees with CPU affinity
The VMkernel handles scheduling the many virtual CPUs (vCPUs) assigned to the VMs among the smaller number of cores in the ESXi server. This behavior enables CPU overcommit. Overcommit is a central part of the cost savings that virtualization delivers. Every few milliseconds the VMkernel looks at what vCPUs it has on each core and chooses whether to change which vCPU it is running. The workloads in the VMs have their own peaks and troughs of load. The VMkernel must be able to move vCPUs from one core to another, potentially running any vCPU on any core. The VMkernel will try to use every core to deliver all of the CPU time the VMs want.
The main misconception about CPU affinity is that it dedicates a core to a particular VM. This is usually why an administrator would incorrectly apply CPU affinity to a critical VM that is heavily dependent on CPU performance. This will often result in lower performance for the VM. CPU affinity does not guarantee cores; rather it denies all other cores to this VM. Currently there is no way for vSphere to guarantee particular cores to a VM. The only guarantee we have is a reservation. A reservation is a guarantee to make a resource available rather than dedicating the resource. For a VM with a critical CPU requirement you should set a CPU reservation sufficient to meet the VMs SLA. Setting CPU affinity on a VM will never increase CPU performance to that VM. In fact, it will do quite the opposite.
The objective of CPU affinity is to remove some of the flexibility from the VMkernel CPU scheduler. CPU affinity restricts the cores that may be used for a VMs vCPUs. Any time we restrict the flexibility of the VMkernel we reduce efficiency and put performance at risk. An ESXi server with 12 cores means there are 12 places the VMkernel can run vCPUs. If a VM has CPU affinity set for cores 2 and 3, then the other 10 cores cannot be used to run this VM. Even if the other 10 cores are all idle, the VMkernel cannot use them for this VM. Even worse is if two busy VMs have CPU affinity set to the same core. Then the VMs will be forced to fight over that one core even when other cores are idle.
Additional issues with CPU affinity
Another consequence of CPU affinity is it disables vMotion. CPU affinity dictates the use of specified cores in an ESXi server for the VM. Since those physical cores cannot move between hosts, neither can the VM that must run on those cores. This has big implications in a Distributed Resource Scheduler (DRS) cluster. This is where vCenter vMotions VMs between ESXi servers to deliver peak performance.
VMs that cannot be vMotioned lead to unbalanced clusters and again can lead to performance problems. VMotioning a lot of VMs is also part of any ESXi server maintenance, so VMs that cannot be moved will cause operational problems. Even worse, if you set a DRS cluster into fully automated mode, then the CPU affinity user interface disappears. Any CPU affinity set is still applied and you cannot remove the setting.
While setting CPU affinity prevents VMotion, it does not prevent HA failover. A VM with CPU affinity will be started on one of the other hosts in the HA cluster and will retain its CPU affinity settings on the new host.
Uses for CPU affinity
What is CPU affinity good for? I use it to cause artificial CPU contention between two VMs. If you set both VMs to use the same cores, they can saturate the cores without saturating the ESXi server. This is not a real world use, but it is a helpful lab technique.
In the real world, I have only heard of one use of CPU affinity; a student supported an old application in a VM. The application is licensed to the serial number of the CPU core where it runs. This is a truly awful licensing scheme. The only way to support the application under virtualization is to tie it to one core with CPU affinity.
Should you use CPU affinity in Windows Server 2012 R2?