VMware's VMotion technology is the process by which a running Virtual
Machine is migrated from one ESX host to another without incurring
downtime. This how-to will explore how to obtain VMotion, how to
configure it, what its appropriate usage scenarios are, and finally,
how VMotion technology actually works.
How to obtain VMotion
VMotion is a component of VMware's Virtual Infrastructure 3 (VI3)
enterprise virtualization solution. VMware offers three VI3 editions --
Starter, Standard, and Enterprise. VMotion is available a la carte for
the Starter and Standard editions and comes with the Enterprise edition.
For more information on pricing, call a VMware Sales
Representative.
VMware licenses its products per every two processors. Notice the use
of the word "processors," not "cores." For the moment, VMware is still
licensing their software per processor, as long as the processor has
only four cores (more information at the VMware Website). This
could of course change in the future. There are two different ways to
go about purchasing VI3. It is possible to purchase it directly from
VMware Sales or through an authorized VMware Reseller. The VMware Sales team
or an an authorized VMware Reseller will have detailed information about
the actual purchase process.
Regardless of the avenue by which the software is obtained, it is
necessary to create a VMware account. This process begins here at
the VMware Website. This account will
be linked with all purchased VMware licenses. It may be a good idea to
create a group or user agnostic email address to create this account
with in order to eliminate a single point of failure. Once the account
is created and the VI3 Enterprise edition has been purchased, it is
possible to obtain the license files needed by ESX and most importantly
with regards to this article, VMotion.
Unfortunately for the customer, VMware's actual implementation of its
licensing does not break down as nicely as its three VI3 editions. Once
VI3 has been purchased, VMware will send the buyer several emails with
licensing information. These e-mails will include the License
Activation Code(s) that can be used to register the product(s) with
VMware. Visit this VMware page to
register the product(s). Once the product(s) is/are registered it is
possible to activate the software license(s) at the license management page. There is no nice way to explain how
to create the license file. The user interface has changed twice now
since this author last activated VI3 licenses. VMware is aware of how
cumbersome the licensing process is and provides online VI3 Licensing
Specialists that will assist a user via a live chat session.
Once the license file has been created and obtained by
downloading via e-mail, it is time to install it the file into
the licensing server. VI3 provides its own licensing server, but it is
also possible to install the license file into an existing FlexLM
environment. It would be pointless to repeat here how to install the
license file when VMware has already explained it in great detail and
quite nicely on page 43 of the VI3 Installation Guide. When installing
the license file, please be cognizant of whether or not all the VI3
licenses were combined into one license file or there were several
license files received. Although VI3's components, VMware Distributed
Resource Scheduler (DRS), High Availability (HA), VMotion,
VirtualCenter, and ESX all exist under the aegis of VI3, each of their
licenses can be obtained as single files. This used to be the way it
was handled, but now VMware allows the combination of the separate
licenses into a single file on their web site. If the licenses are not
combined on the web site, it will be necessary to combine them into a
single file before installing them into the license server.
Once the license file is installed, including the VMotion license,
VMotion must be configured.
Configuring VMotion
Once all the licensing has been worked out, it is time to "turn on"
VMotion. This assumes that ESX and VirtualCenter have already been
installed and properly configured. Launch the VI3 Client and click on
one of the ESX hosts that should have VMotion enabled. Once the host is
selected click on the "Configuration" tab. Click on the link "Licensed
Features" inside of the box labeled "Software." There is a section
called "Add-Ons." Click on the link "Edit..." that is grouped with the
"Add-Ons" section. If VI3 licensing has been properly configured then
it should be possible to select the checkbox next to "VMotion". Click
"OK" to enable VMotion for this host. Enabling VMotion on one host by
itself is actually rather useless. The above steps should be repeated
for each host that will have VMs migrated to or from it.
VMotion requires a VMotion-enabled network to function. Although it is
not required, it is *strongly* recommended that VMotion have a dedicated
NIC. In fact, if a VMotion is not functioning correctly and by all
appearances should be, the problem will most likely be network related.
VMotion sends a lot of information over the wire and needs plenty of
room and speed to do so. Regardless of whether or not a NIC can be
dedicated for VMotion, a port must be enabled for VMotion. Launch the
VI3 Client and click on one of the ESX hosts that should have VMotion
enabled. Once the host is selected click on the "Configuration" tab.
Click on the link "Networking" inside of the box labeled "Hardware."
Click on on the "Add Networking..." link in the upper right-hand corner
of the VI3 client. Select "VMKernel" for the connection type and click
"Next." If there is a dedicated NIC available to VMotion then select
"Create a virtual switch" otherwise select an existing vswitch and click
"Next". On the next screen label the port group "VMotion" and click on
the checkbox that says "Use this port group for VMotion." Do *not* fill
out the VLAN ID field. Fill out the IP Address, Subnet Mask fields. Do
not fill out the VMKernel Default Gateway field. Click "Next" to
continue and then click "Finish" to create the port group. The above
steps should be repeated for each host that will have VMs migrated to or
from it.
Now that both licensing and the VMotion networks are configured, it is
possible to migrate VMs from one host to another.
VMotion usage scenarios
Now that VMotion is enabled on two or more hosts, when should it be
used? There are two primary reasons to use VMotion: to balance the load
on the physical ESX servers and eliminate the need to take a service
offline in order to perform maintenance on the server.
VI3 balances its load by using a new feature called DRS. DRS is
included in the VI3 Enterprise edition along with VMotion. This is
because DRS uses VMotion to balance the load of an ESX cluster in real
time between all of the server involved in the cluster. For information
on how to configure DRS see page 95 of the VMware VI3 Resource
Management Guide.
Once DRS is properly configured it will constantly be evaluating how
best to distribute the load of running VMs amongst all of the host
servers involved in the DRS-enabled cluster. If DRS decides that a
particular VM would be better suited to run on a different host then it
will utilize VMotion to seamlessly migrate the VM over to the other host.
While DRS migrates VMs here and there with VMotion, it is also possible
to migrate all of the VMs off of one host server (resources permitting)
and onto another. This is accomplished by putting a server into
"maintenance mode." When a server is put into maintenance mode, VMotion
will be used to migrate all of the running VMs off it onto another
server. This way it is possible to bring the first server offline to
perform physical maintenance on it without impacting the services that
it provides.
How VMotion works
As stated above, VMotion is the process that VMware has invented to
migrate, or move, a virtual machine that is powered on from one host
server to another host server without the VM incurring downtime. This
is known as a "hot-migration." How does this hot-migration technology
that VMware has dubbed VMotion work? Well, as with everything, in a
series of steps:
- A request has been made that VM-A should be migrated (VMotioned) from ESX-A to ESX-B
- VM-A's memory is pre-copied from ESX-A to ESX-B while ongoing changes
are written to a memory bitmap on ESX-A.
- VM-A is quiesced on ESX-A and VM-A's memory bitmap is copied to ESX-B.
- VM-A is started on ESX-B and all access to VM-A is now directed to
the copy running on ESX-B.
- The rest of VM-A's memory is copied from ESX-A all the while memory
is being read and written from VM-A on ESX-A when applications attempt
to access that memory on VM-A on ESX-B.
- If the migration is successful VM-A is unregistered on ESX-A.
For a VMotion event to be successful the following must be true:
*Editor's Note: Special thanks to Colin Stamp of IBM United Kingdom Ltd. for rewriting the following list.
- The VM cannot be connected to an internal vswitch.
- The VM cannot be connected to a CD-ROM or floppy drive that is using an ISO or floppy image stored on a drive that is local to the host server.
- The VM's affinity must not be set, i.e., binding it to physical CPU(s).
- The VM must not be clustered with another VM (using a cluster service like the Microsoft Cluster Service (MSCS)).
- The two ESX servers involved must both be using (the same!) shared storage.
- The two ESX servers involved must be connected via Gigabit Ethernet (or better).
- The two ESX servers involved must have access to the same physical networks.
- The two ESX servers involved must not have virtual switch port groups that are labeled the same.
- The two ESX servers involved must have compatible CPUs. (See support on Intel and AMD).
If any of the above conditions are not met, VMotion is not supported and will not start. The
simplest way to test these conditions is to attempt a manual VMotion
event. This is accomplished by right-clicking on VM in the VI3 client
and clicking on "Migrate..." The VI3 client will ask to which host this
VM should be migrated. When a host is selected, several validation
checks are performed. If any of the above conditions are true then the
VI3 client will halt the VMotion operation with an error.
Conclusion
The intent of this article was to provide readers with a solid grasp of
what VMotion is and how it can benefit them. If you have any
outstanding questions with regards to VMotion or any VMware technology
please do not hesitate to send them to me via ask the experts.
About the author:
Andrew Kutz is deeply embedded in the dark, dangerous world of
virtualization. He is a Microsoft Certified Solutions
Developer (MCSD), a SANS/GIAC Certified Windows Security Administrator
(GCWN), and a VMware Certified Professional (VCP) in VI3. Andrew also
recently submitted a security practical entitled "Sudo for Windows
(sudowin)" to the SANS organization where it was accepted, thereby
granting Andrew SANS GOLD status.
