How to set up a home vSphere lab using a nested hypervisor

A vSphere lab created with a nested hypervisor can be more stable with less overhead than a similar home lab on a hosted hypervisor like Workstation.

Home virtualization labs offer IT pros a chance to try out the latest technologies with the benefit of flexible...

hours and zero risk to production environments.

There's more than one way to build a VMware home lab. One option, the do-it-yourself nested vSphere-in-vSphere lab, includes some portions that are not totally supported by VMware Inc., but it's a way to try out advanced features of vSphere 5.1 with low overhead.

A vSphere 5.1 lab can be extremely useful for administrators studying for their VMware Certified Professional test, said Matt Kozloski, virtualization practice lead at Kelser Corp. and Hartford VMUG leader. He shared his tips on building a vSphere 5.1 home lab with a nested hypervisor during a 2013 Hartford VMUG meeting and offered pointers on resource allocation, shared storage without the vSphere Storage Appliance (VSA) and networking setup.

Building a home lab

A nested hypervisor is a hypervisor contained within a virtual machine (VM). Kozloski created the nested hypervisor style of a vSphere home lab rather than testing vSphere 5.1 in a desktop virtualization tool like VMware Workstation because Workstation adds an extra OS layer and its network address translation (NAT) is sometimes "fussy." Nested hypervisor labs can be more complicated, but that's part of the fun, Kozloski said.

Nested hypervisor labs can be more complicated, but that's part of the fun.

Matt Kozloski

To get your lab up and running, start with the VMware 60-day evaluation license of vSphere 5.1. "There are a lot of free software tools available to you," Kozloski noted. Microsoft TechNet lets IT managers and admins evaluate Microsoft software via a paid subscription, so go here for your guest OSes. DreamSpark, similar to TechNet, offers professional software free to students and academic institutions.

Your vSphere home lab setup must meet 5.1's requirements, and Kozloski recommends using Intel EPT or AMD RVI for 64-bit nested guests.

"Use your hard disk drives for the VMware Virtual Machine File System [VMFS] and install vSphere on a USB stick," he advised, to keep the lab lean. Because the nested home lab is all vSphere, it is very stable, but remember that VMware doesn't support the entire setup. You might run into warnings (see Figure 1), but they won't actually prevent you from using the nested lab.


Figure 1. This error message says that nested virtualization is not supported on the CPU. But you can proceed and the lab will function.

Tech specs of a home lab

The nested vSphere lab requires two CPUs and 2 gigabytes of RAM presented to each host at the start. After the build, you can take away one CPU, but memory must remain at or above 2 GB. Watch out for memory swapping with other apps, Kozloski warned. And remember that management tool vCenter can hog a considerable amount of RAM.

VMware vSphere 5.1, unlike vSphere 5.0, doesn't require you to enable nested hypervisors. However, nested hypervisor support allows the lab to virtualize 64-bit VMs inside your vSphere hosts, so Kozloski suggested taking that step.

To work with shared storage in your home lab, create shared virtual machine disks (VMDKs). The VSA will eat up resources in your lab environment, so Kozloski opts for a shared serial-attached SCSI (SAS) disk approach. Shared SAS avoids overhead and enables a multi-write environment, which VMFS normally prevents. That way, more than one VM can write to the same file.

To save space on solid-state storage drives, you can create what Kozloski calls manual linked clones: "Build a base VMDK, snapshot it, then make however many copies of that snapshot you will use," he said. "These are now delta VMDKs of the parent VMDK. Use the delta VMDKs for new VMs."

The danger with this method is that you can delete the parent VMDK accidentally, so Kozloski suggests removing unwanted information from inventory, then deleting it from the VMFS.

Kozloski recommends a private networking subnet for the nested hypervisor setup. This creates a "portable pod" of networking with virtual switches connecting the physical network and an isolated private bubble network. Platform-independent network router and firewall tools, like those from Vyatta Inc., will control traffic to and from guest virtual machines. To connect the nested VMs, enable promiscuous mode on the virtual switch to the physical network.

There are two good adapter options for networking the nested hypervisor, Kozloski said: The purely virtual VMXNET3 from VMware and the E1000, which emulates Intel's 82545EM Gigabit Ethernet network interface card (NIC). He suggests the E1000 to avoid Purple Screen of Death failures, calling VMware's VMXNET3 NIC too experimental for the nested hypervisor. However, it's safe for the guest VMs of the nested hypervisor.


Figure 2. This is what the host system will look like (A), and how the nested vSphere cluster appears (B).

Images courtesy of Matt Kozloski.



Dig Deeper on VMware how-tos