Tip

How ESX “Load-Balances” and Fail-Over/Back Bonds/NIC teams

One popularly misunderstood feature of ESX Bonds/Teams is how it handles “load-balancing” and fail-over and fail-back. You should know the default method for a Bond is “out-mac” and that the “load-balancing” is done by MAC address. The term “out-mac” comes from the entry in the configuration file that enables this.

 

Out-MAC

  • When a Virtual NIC is activated within a Virtual machine the vmKernel sends an ARP request for virtual MAC for the adapter down one of the physical paths for the virtual switch
  • The algorithm for doing this does not consider network load. So it is entirely possible that it could select a heavily loaded NIC.
  • This default behavior can reduce the benefits of load-balancing
  • Requires no special configuration on the physical switch
  • vmkernal controls the “advertising” of the MAC address, so there is no danger of the “duplicate MAC address” conflicts
  • You can plug one NIC in a bond to one physical switch, and the second NIC into another physical switch and it will work.  This increases the redundancy at the physical level
  • If one physical link fails, the vmkernal should automatically re-arp the VM’s MAC address down the alternative link. This occurs within 10 seconds or 0-4 ICMP timeouts, and often much sooner
  • When the primary link comes back online it takes a large number of timeouts to fail-back to it. This is due to the Spanning-Tree protocol

    Requires Free Membership to View

  • that is enabled on most switches
  • Given all this, MAC is great for redundancy when network traffic is not great. If however, bandwidth is a premium you may decide to change this to the IP method of load-balancing the traffic
     

Out-IP

  • Provides for a more balanced configuration than MAC address
  • Fail-over works the same way, as with fail-back
  • As the MAC address of the virtual NICs will be seen by multiple physical switches, your switches will need additional configuration. Without changing the configuration, you will face issues of duplicate MAC addresses which will cause switching problems
  • This is achieved by using EtherChannel on Cisco equipment or LACP (Link Aggregation Control Protocol) on the physical switch
  • In addition, you will need to edit a file on the ESX server to switch from MAC address to IP address method:
  1. Logon to the Service Console as Root
  2. nano -w /etc/vmware/hwconfig
  3. locate the niceteam.bondn value related to the friendly name of your NIC

    Change the value 

     

    nicteam.bondn.load_balance_mode = “out-mac”

    to

    nicteam.bondn.load_balance_mode = “out-ip”

Standby Mode

  • There is no load-balancing mode associated with this method
  • It merely allows you to set your preferred vmnicn in a bond/team. If it fails, then the system uses the next NIC in the bond/team
  • Fail-over/back is achieved by the same method as MAC/IP methods
  • Some people set the preferred to be a 1GB card, and the second adapter to be a 100MB card
  • Personally, all my NICs are GB anyway, so it doesn’t really matter which one it uses
  • Making the change below disables all load-balancing that was previously discussed
  1. Logon to the Service Console as Root
  2. nano -w /etc/vmware/hwconfig
  3. Add the following entry:nicteam.bondn.home_link = “vmnic1″

 

This was first published in October 2005

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.