James Thew - Fotolia


VM Encryption enhances security in VMware vSphere 6.5

Although VM-level encryption has been around for some time, it's been riddled with problems. VMware aims to change all that with the new VM Encryption tool in vSphere 6.5.

In spite of its ability to secure VMs against unauthorized use, VM-level encryption has been a slow starter, in large part due to bugs in the system. VMware hopes to smooth out these kinks with its own VM Encryption tool, included in vSphere 6.5. VSphere 6.5 VM Encryption doesn't occur within the guest OS, but rather at the hypervisor of Virtual Machine File System level. This way, there's nothing to install in the guest OS in either Windows or Linux.

How does VM Encryption work?

VM Encryption is managed via a storage policy. VSphere 6.5 has a Storage Policy-Based Management system that you can use to create policies that define storage requirements for virtual disks and the configuration files of a VM.

Under the VM Storage Policies tab, shown in Figure A, you can modify the sample storage policy for VM Encryption and apply it to VM objects, such as virtual disks.

VM Encryption Policy.
Figure A. Create a custom VM Encryption policy.

In order to configure VM Encryption, you must first add a key management server (KMS), as this isn't built-in; you can choose from any of the available -- and free -- products out there. Select your vCenter Server and follow this chain: Manage > Key Management Services > Add Server.

When the I/O operation comes up from the VM to the virtual disk controller, it's encrypted by a kernel module, which is part of the ESXi hypervisor. Then, the I/O alone is sent to the storage layer.

What does 'encrypted' mean?

All VM files -- including Virtual Machine Disk files, virtual machine executable (VMX) configuration files, snapshot files and VMX swap files -- are stored in folders. All files stored in folders are encrypted.

Encryption is managed by the hypervisor, rather than the guest VM, which means the keys are not exploitable through the VM's memory.

VM Encryption with vMotion

You can also set up VM-level encryption on vMotion. This means when you configure VM-level encryption and then initiate a vMotion migration, vCenter Server will generate a random 256-bit key; this key is then packaged and sent to the hosts used in the vMotion operation. VMotion data uses this key to encrypt data.

You have the option of using traditional or encrypted vMotion on nonencrypted VMs, but note that you can only use encrypted vMotion on encrypted VMs.

This occurs at the VM hardware level. As you can see in Figure B, you'll be given the option to set the encrypted vMotion as Disabled, Opportunistic or Required. The Disabled option doesn't use encryption for vMotion. The Opportunistic option uses the encrypted vMotion if the destinations host supports it, otherwise, it uses the nonencrypted vMotion. As it implies, the final option, Required, requires you to use encrypted vMotion. The system will run a check to see if the destination host -- vSphere 6.5 -- supports encryption; if it doesn't, vMotion will fail.

Encrypted vMotion options.
Figure B. Encrypted vMotion options.

The process of configuring VM Encryption is quite simple. If you already have a KMS set up, all you need to do is direct your vCenter Server to it, create an encryption policy and apply to selected VMs. Encrypting vMotion traffic is also simple, but requires the latest ESXi 6.5 on both ends.

Next Steps

VMware comes out of 2016 on top

What's new in vSphere 6.5?

How to create an encrypted VM in Workstation

Dig Deeper on Securing a VMware environment