Tip

PowersHell – The Uber/Master vSwitch

Some customers I know like to create on each ESX host what they call the uber-vSwitch or the Master-vSwitch. I sometime jokingly call it the “Lord of the Rings vSwitch” – One vSwitch to rule them all!

The concept is simple. Sometimes they have a lack of vmnics (limited PCI bus) or a lack of physical switch ports – so their ESX host only has 4-nics. If you are doing Management, VMotion, iSCSI/NAS, HA-Heartbeat and FT-Logging – it's quite tricky to keep all of these separate (despite the use of VLANs) whilst offering redundancy. So some of my customers opt for the UbervSwitch or MastervSwitch option. It looks like this:

    Requires Free Membership to View

As you can see fault-tolerance is offered by attaching all the NICs to the vSwitch. Each type of network activity resides on different VLAN (Production/10, VMotion&HA-Heartbeat, 11 and IP-Storage/12). Of course the worry here is that the bandwidth intensive nature of VMotion or IP-Storage could affect your VMs and Management of the ESX host. So that leads people down the route in some cases of using Active/Passive on the Portgroup level to try and control the traffic a little bit more…

As you can see the preferred VMNIC for the “Production” portgroup is vmnic1, with the other nics being standby. This configuration could be repeated so the preferred VMNIC for VMotion/HA-Heartbeat could be vmnic2, with the remain NICs being standby. This way you can have the best of both worlds. Network separation (different traffic types prefer its particular VMNIC) but if you have a VMNIC failure there is still redundancy through all the network dependent components.

So I got to thinking how this quite complicated configuration would take ages to do by hand, and quite a challenge to do using the esxcfg-commands or the RCLI/vMA. So I worked out how to do this using powersHell. 

# Set Variables:
# This version just has all the variables at the top.
# These variables are HOST specific...
$vmhost = "esx4.vi4book.com"
$VMotionIP = "10.0.0.204"
$HAheartbeatIP = "10.0.0.104"
$IPstorageIP = "172.168.3.104"

# Adds vmnic1/2/3 to vSwitch0 and creates the portgroups of Production, VMotion, Ip-Storage, HA-Heartbeat

$vs0 = Get-VirtualSwitch -Name vSwitch0
Set-VirtualSwitch -VirtualSwitch $vs0 -Nic vmnic1, vmnic2, vmnic3

#Add the Production Port Group to vSwitch0

$Production =  New-VirtualPortGroup -VirtualSwitch $vs0 -Name Production

# Creates a portgroup for VMotion on vSwitch0

New-VMHostNetworkAdapter  -PortGroup VMotion -VirtualSwitch $vs0 -IP $VMotionIP -SubnetMask 255.255.255.0 -VMotionEnabled: $true

# This method will create a vswif interface. -ConsoleNIC ONLY works with ESX "Classic". There is currently no way to set "Management Traffic" on a VMKernel Port in ESXi
# PortGroups added to vSwitch0

$HAheartbeat = New-VirtualPortGroup -VirtualSwitch $vs0 -Name HA-Heartbeat
# New-VMHostNetworkAdapter -PortGroup HA-Heartbeat -VirtualSwitch $vs0 -IP $HAheartbeatIP -SubnetMask 255.255.255.0 -ConsoleNic
New-VMHostNetworkAdapter -PortGroup HA-Heartbeat -VirtualSwitch $vs0 -IP $HAheartbeatIP -SubnetMask 255.255.255.0

# This creates a VMKernel Port Group on vSwitch3 called IP-Storage
$IPstorage = New-VirtualPortGroup -VirtualSwitch $vs0 -Name IP-Storage
New-VMHostNetworkAdapter  -PortGroup IP-Storage -VirtualSwitch $vs0 -IP $IPstorageIP -SubnetMask 255.255.255.0

# By default all portgroups would default to originating port id, and the traffic could go through any nic.
# This could mean your VMotion traffic could go on the same NIC as your storage traffic
# Using active & standby on portgroups would allow us to stop this..
# Script assumes you have ESXi and using a management port group called "Management Network"
# Replace "Management Network" with Service Console if your using ESX "Classic"
# Set VLAND ID as you see appropriate

# Management Network (vmnic0 - Active, vmnic1,2,3 - standby)
$vSwitch = "vSwitch0"
$esxhost = Get-VMHost $vmhost
$hostview = $esxhost | Get-View
$ns = Get-View -Id $hostview.ConfigManager.NetworkSystem

$pgspec = New-Object VMware.Vim.HostPortGroupSpec
$pgspec.vswitchName = "vSwitch0"
$pgspec.Name = "Management Network"
$pgspec.vlanId = "0"
$pgspec.Policy = New-Object VMware.Vim.HostNetworkPolicy
$pgspec.Policy.NicTeaming = New-Object VMware.Vim.HostNicTeamingPolicy
$pgspec.Policy.NicTeaming.nicOrder = New-Object VMware.Vim.HostNicOrderPolicy
$pgspec.Policy.NicTeaming.nicOrder.activeNic = @("vmnic0")
$pgspec.Policy.NicTeaming.nicOrder.standbyNic = @("vmnic1","vmnic2","vmnic3")
$ns.UpdatePortGroup($pgspec.Name,$pgspec)

# Production (vmnic1 - Active, vmnic0,2,3 - standby)
$vSwitch = "vSwitch0"
$esxhost = Get-VMHost $vmhost
$hostview = $esxhost | Get-View
$ns = Get-View -Id $hostview.ConfigManager.NetworkSystem

$pgspec = New-Object VMware.Vim.HostPortGroupSpec
$pgspec.vswitchName = "vSwitch0"
$pgspec.Name = "Production"
$pgspec.vlanId = "10"
$pgspec.Policy = New-Object VMware.Vim.HostNetworkPolicy
$pgspec.Policy.NicTeaming = New-Object VMware.Vim.HostNicTeamingPolicy
$pgspec.Policy.NicTeaming.nicOrder = New-Object VMware.Vim.HostNicOrderPolicy
$pgspec.Policy.NicTeaming.nicOrder.activeNic = @("vmnic1")
$pgspec.Policy.NicTeaming.nicOrder.standbyNic = @("vmnic0","vmnic2","vmnic3")
$ns.UpdatePortGroup($pgspec.Name,$pgspec)

# VMotion (vmnic2 - Active, vmnic0,1,3 - standby)
$vSwitch = "vSwitch0"
$esxhost = Get-VMHost $vmhost
$hostview = $esxhost | Get-View
$ns = Get-View -Id $hostview.ConfigManager.NetworkSystem

$pgspec = New-Object VMware.Vim.HostPortGroupSpec
$pgspec.vswitchName = "vSwitch0"
$pgspec.Name = "VMotion"
$pgspec.vlanId = "11"
$pgspec.Policy = New-Object VMware.Vim.HostNetworkPolicy
$pgspec.Policy.NicTeaming = New-Object VMware.Vim.HostNicTeamingPolicy
$pgspec.Policy.NicTeaming.nicOrder = New-Object VMware.Vim.HostNicOrderPolicy
$pgspec.Policy.NicTeaming.nicOrder.activeNic = @("vmnic2")
$pgspec.Policy.NicTeaming.nicOrder.standbyNic = @("vmnic0","vmnic1","vmnic3")
$ns.UpdatePortGroup($pgspec.Name,$pgspec)

# VMotion (vmnic2 - Active, vmnic0,1,3 - standby)
$vSwitch = "vSwitch0"
$esxhost = Get-VMHost $vmhost
$hostview = $esxhost | Get-View
$ns = Get-View -Id $hostview.ConfigManager.NetworkSystem

# HA-Heartbeat same network as VMotion...
$pgspec = New-Object VMware.Vim.HostPortGroupSpec
$pgspec.vswitchName = "vSwitch0"
$pgspec.Name = "HA-Heartbeat"
$pgspec.vlanId = "11"
$pgspec.Policy = New-Object VMware.Vim.HostNetworkPolicy
$pgspec.Policy.NicTeaming = New-Object VMware.Vim.HostNicTeamingPolicy
$pgspec.Policy.NicTeaming.nicOrder = New-Object VMware.Vim.HostNicOrderPolicy
$pgspec.Policy.NicTeaming.nicOrder.activeNic = @("vmnic2")
$pgspec.Policy.NicTeaming.nicOrder.standbyNic = @("vmnic0","vmnic1","vmnic3")
$ns.UpdatePortGroup($pgspec.Name,$pgspec)

# IP Storage (vmnic3 - Active, vmnic0,1,2 - standby)
$pgspec = New-Object VMware.Vim.HostPortGroupSpec
$pgspec.vswitchName = "vSwitch0"
$pgspec.Name = "IP-Storage"
$pgspec.vlanId = "12"
$pgspec.Policy = New-Object VMware.Vim.HostNetworkPolicy
$pgspec.Policy.NicTeaming = New-Object VMware.Vim.HostNicTeamingPolicy
$pgspec.Policy.NicTeaming.nicOrder = New-Object VMware.Vim.HostNicOrderPolicy
$pgspec.Policy.NicTeaming.nicOrder.activeNic = @("vmnic3")
$pgspec.Policy.NicTeaming.nicOrder.standbyNic = @("vmnic0","vmnic1","vmnic2")
$ns.UpdatePortGroup($pgspec.Name,$pgspec)

# Removes "VM Network" from the vSwitch0

get-VirtualPortGroup  | where { $_.Name -like "VM Network"} |  Remove-VirtualPortGroup  -Confirm:$false

 

This was first published in August 2009

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.