James Thew - Fotolia


Set up VMware VM Encryption for hypervisor-level security

Learn how to set up and run the new VMware vSphere 6.5 VM Encryption feature to improve security by bringing encryption to the hypervisor level.

The new VM Encryption tool in vSphere 6.5 goes beyond standard VM-level security by performing the encryption at...

the hypervisor level.

Doing the encryption at the hypervisor level instead of in the virtual machine makes encryption agnostic, as well as policy-driven -- VMware VM Encryption is managed via storage policy. While the main purpose of the tool is to enhance security, it means encryption is no longer an all-or-nothing proposal; encryption of the storage area network or underlying storage are no longer the only options.

There is a small overhead for VM Encryption, though this is to be expected in any encryption system. For what it's worth, I hardly noticed the overhead during my experimentation with ESXi encryption.

Implementing VM Encryption is quite simple. You can easily set up a basic proof of concept (POC) implementation for the encryption infrastructure, as VMware has designed the underlying cryptographic system to use third-party plugins. VMware's current list of approved vendors includes RSA and Symantec, as well as several others. There is currently no VMware implementation.

Before implementing the encryption system, it's important to understand how VM Encryption works. Put simply, the encryption is handed from the encryption VM to its client, the vCenter.

Setting up the cryptographic back end is straightforward. Most vendors will ship a VM appliance that can be installed, powered on and configured. Since configuring the cryptography for each of the vendor plugins is beyond the scope of this guide, I simply used the modified POC encryption manager that VMware released to beta testers.

This test version only keeps encryption keys for the duration of the machine's uptime. This is just a demonstration system and the encryption key will be lost upon reboot, so don't encrypt any machines you actually use. The best way to avoid running into trouble is to create a couple of test VMs.

Before implementing the encryption system, it's important to understand how VM Encryption works. Put simply, the encryption is handed from the encryption VM to its client, the vCenter. The vCenter then provides keys as needed to the ESXi hosts. These are stored in a secure manner to enable you to unlock the VM. The keys are never written to the disk on the ESXi host. However, the intermediate keys for locking and unlocking the VMs are stored in a secure encryption enclave.

To set up the encryption server, you need to set up a Linux host with Docker. Once you've done this, pull down the Docker image and run the instance with the following command:

sudo docker pull lamw/vmwkmip

sudo docker run --rm -it -p 5696:5696 lamw/vmwkmip

At this point the Docker image should be running on port 5696.

Setting up the cryptographic infrastructure

Now that we've covered how encryption is applied, let's look at how to set up the infrastructure. First, add the Key Management Service (KMS) server to the vCenter by going to the top level of the vCenter configuration menu and selecting Key Management Servers from the hyperlink on the left.

Configure the KMS server.
Figure A. Configure the Key Management Service server.

This will bring up a dialog box that allows you to enter KMS server details. The exact details will vary, and some KMS server configurations may require a username and password. We don't need to use one in this instance. The server address should be that of the Ubuntu server. The port used for this example is 5696.

KMS server details.
Figure B. KMS server details.

Once you've submitted the KMS details, you'll be prompted to accept a certificate; accept this, and KMS will be set to the default.

At this point, the cryptographic configuration is complete. However, a single KMS is a single point of failure, therefore, I recommend configuring a minimum of two. Do not encrypt the vCenter or it will prevent vCenter from booting. You need to avoid this because vCenter is a key component of the cryptographic infrastructure.

The next step is to create an encryption storage policy. If you navigate to VM Storage Policies, you'll see a new storage policy titled VM Encryption Policy. There are several options that you can modify if necessary.

VM Storage Policy options.
Figure C. VM Storage Policy options.

At this point, you can encrypt the VM. It's best practice to only encrypt the disks; it's possible to encrypt other items, but it's unadvisable unless you have an overriding reason to do so.

Encrypting VMs

From here on out, encrypting the VM is clear-cut. Before encrypting a device, you must first power it off. Navigate to the VM you plan on encrypting and right-click to edit its setting and expand the disks to encrypt. Select the VM Encryption Policy from the VM storage policy drop-down menu.

Disabling VM Encryption is as easy as changing the policy to the default data store policy. Again, you must power off the VM to perform the necessary actions.

There are a few caveats to using VM Encryption. For one, it does not support exporting encrypted VMs to open virtualization format. The use of cryptography on a per VM basis is dependent on the business and security requirements of the company in question. Exercise caution when implementing the encryption and make sure you fully understand and recognize the ramifications and functionality of encryption.

Next Steps

NSX leads the pack for VMware security

Explore ESXi hypervisor security features

What's next in the world of cryptography?

How does VMware vMotion encryption work?

Dig Deeper on Securing a VMware environment