Comments on: HP Blades, Virtual Connect & ESX considerations http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/ Philip Sellers' random thoughts on technology Mon, 30 Jan 2012 15:01:36 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Bobby McIntire http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/comment-page-1/#comment-578 Bobby McIntire Wed, 29 Apr 2009 00:54:16 +0000 http://tech.philipsellers.com/?p=368#comment-578 Philip/Ray, I guess I'm a little confused. I had my enclosure set up to use Tunnel VLAN tags and we were unable to get a VLAN 100 to pass and communicate correctly. I thought it had to do with the single vlan per nic as described above. According to the network guys, the Tunnel Uplinks that are set up to each Virtual connect in the back of the Enclosure should pass all VLAN. I guess I'm not sure how it should be set up. The current enclosure was working well but we only have a few vlans. I brought up a new server on the 100 vlan and it couldn't communicate. Any help would be appreciated. Philip/Ray,
I guess I’m a little confused. I had my enclosure set up to use Tunnel VLAN tags and we were unable to get a VLAN 100 to pass and communicate correctly. I thought it had to do with the single vlan per nic as described above. According to the network guys, the Tunnel Uplinks that are set up to each Virtual connect in the back of the Enclosure should pass all VLAN. I guess I’m not sure how it should be set up. The current enclosure was working well but we only have a few vlans. I brought up a new server on the 100 vlan and it couldn’t communicate. Any help would be appreciated.

]]> By: Philip http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/comment-page-1/#comment-575 Philip Thu, 23 Apr 2009 17:43:55 +0000 http://tech.philipsellers.com/?p=368#comment-575 Ray - Thanks! I think I have figured out where we ran into problems in our configuration. We had a shared uplink set configured and were attempting to use it instead of simply defining an Ethernet network. I re-read the documentation (btw, HP released a brand new cookbook this week - <a href="http://tr.im/jwf5" rel="nofollow">tr.im/jwf5</a>), and I now see that we should omit the shared uplink for ESX to work correctly in Tunnel mode. Ray – Thanks! I think I have figured out where we ran into problems in our configuration. We had a shared uplink set configured and were attempting to use it instead of simply defining an Ethernet network. I re-read the documentation (btw, HP released a brand new cookbook this week – tr.im/jwf5), and I now see that we should omit the shared uplink for ESX to work correctly in Tunnel mode.

]]> By: Ray http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/comment-page-1/#comment-574 Ray Thu, 23 Apr 2009 16:08:18 +0000 http://tech.philipsellers.com/?p=368#comment-574 Enabling Tunnel VLAN Tags (even according to the Virtual Connect help file) will pass all VLAN tags through the VC domain unmodified. It's ESX that will then manage the various VLAN traffic dependent on your virtual switch configuration. If you have VLAN tunneling enabled and attempt to configure shared uplinks, only untagged frames will be transmitted (tagged frames will be dropped). If you enable Map VLANs option, any dedicated links (ie individual ports that are not members of shared uplink sets) can transmit untagged frames only - all tagged VLAN traffic on dedicated links will be dropped. And you'll have to map the traffic from the shared uplink sets to the appropriate NICs/server profiles. This is a lot of work - ESX will do it for you if you just pass the traffic with VLAN Tunneling (tagging - 802.1q). Check out the HP Virtual Connect Cookbook: http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01471917/c01471917.pdf Scenarios 7 & 8 specifically detail ESX config. Enabling Tunnel VLAN Tags (even according to the Virtual Connect help file) will pass all VLAN tags through the VC domain unmodified. It’s ESX that will then manage the various VLAN traffic dependent on your virtual switch configuration. If you have VLAN tunneling enabled and attempt to configure shared uplinks, only untagged frames will be transmitted (tagged frames will be dropped).
If you enable Map VLANs option, any dedicated links (ie individual ports that are not members of shared uplink sets) can transmit untagged frames only – all tagged VLAN traffic on dedicated links will be dropped. And you’ll have to map the traffic from the shared uplink sets to the appropriate NICs/server profiles. This is a lot of work – ESX will do it for you if you just pass the traffic with VLAN Tunneling (tagging – 802.1q).
Check out the HP Virtual Connect Cookbook:
http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01471917/c01471917.pdf
Scenarios 7 & 8 specifically detail ESX config.

]]> By: Philip http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/comment-page-1/#comment-571 Philip Tue, 21 Apr 2009 17:51:36 +0000 http://tech.philipsellers.com/?p=368#comment-571 Bobby, No, I think you'll be able to pass as many VLANs as you configure onto the 3 physical ports connected to vSwitch2, but you'll need to be in map vLAN mode. When you change your Virtual Connect option, it exposes a "Multiple VLANs" selection which lets you choose which VLANs you want to carry across those ports. You have the capability, you just need a small configuration change. This configuration change does not affect any of your other server profiles that are already created. It just exposes the "Multiple VLANs" capability in a server profile. In tunnel VLAN mode, you are only capable of carrying a single VLAN at a time (as far as I can find anywhere). I wouldn't recommend tagging 3 different VLANs, one on each physical port and then using all 3 in the same vSwitch. I don't think that's a good practice in VMware. I'm not sure that scenario would work at all. Bobby,
No, I think you’ll be able to pass as many VLANs as you configure onto the 3 physical ports connected to vSwitch2, but you’ll need to be in map vLAN mode. When you change your Virtual Connect option, it exposes a “Multiple VLANs” selection which lets you choose which VLANs you want to carry across those ports. You have the capability, you just need a small configuration change. This configuration change does not affect any of your other server profiles that are already created. It just exposes the “Multiple VLANs” capability in a server profile.

In tunnel VLAN mode, you are only capable of carrying a single VLAN at a time (as far as I can find anywhere). I wouldn’t recommend tagging 3 different VLANs, one on each physical port and then using all 3 in the same vSwitch. I don’t think that’s a good practice in VMware. I’m not sure that scenario would work at all.

]]> By: Bobby McIntire http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/comment-page-1/#comment-569 Bobby McIntire Tue, 21 Apr 2009 14:44:44 +0000 http://tech.philipsellers.com/?p=368#comment-569 Philip, Just to clarify. We have 6 virtual connect ethernet modules set up with 1 set of fiber connected to a Cisco switch as port channels. Each port channels are set to pass all VLANs. On the ESX side I have 6 nics in a BL460 server. 2 in one Virtual Switch 1 for Service console and vmotion and 3 in Virtual Switch 2 for all Virtual machines and 1 in Virtual Switch 3 for ISCSI (iso and templates). Virtual connect is set up to use Tunnel VLAN Tags. Will we be only able to pass 3 vlans since there are 3 nics on the Virtual Switch 2 running all the virtual machines? We have over 15 VLANS that need to be passed for thru the Virtual Switch 2. Great blog, Bobby McIntire Philip,
Just to clarify. We have 6 virtual connect ethernet modules set up with 1 set of fiber connected to a Cisco switch as port channels. Each port channels are set to pass all VLANs. On the ESX side I have 6 nics in a BL460 server. 2 in one Virtual Switch 1 for Service console and vmotion and 3 in Virtual Switch 2 for all Virtual machines and 1 in Virtual Switch 3 for ISCSI (iso and templates). Virtual connect is set up to use Tunnel VLAN Tags. Will we be only able to pass 3 vlans since there are 3 nics on the Virtual Switch 2 running all the virtual machines? We have over 15 VLANS that need to be passed for thru the Virtual Switch 2.

Great blog,

Bobby McIntire

]]> By: Philip http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/comment-page-1/#comment-509 Philip Tue, 10 Mar 2009 17:59:00 +0000 http://tech.philipsellers.com/?p=368#comment-509 M - That would be my guess. We tried to get multiple VLANs across a single NIC with Tunnel mode, but it would not work on our enclosures. We had to enable the Map VLANs mode to get multiple VLANs across a single NIC. Bringing a single VLAN per NIC to the ESX host will certainly work. We have other clusters where we are doing that with traditional rack-mount servers. In a simple network where all your VM's are on the same VLAN, it may make sense to do this on blades, too. Our deployment is more complex with many VLANs for our VM's. In a blade configuration with 6 total NICs possible, we would be limiting our capabilities if we did not do the tagging. Hope this helps... M – That would be my guess. We tried to get multiple VLANs across a single NIC with Tunnel mode, but it would not work on our enclosures. We had to enable the Map VLANs mode to get multiple VLANs across a single NIC. Bringing a single VLAN per NIC to the ESX host will certainly work. We have other clusters where we are doing that with traditional rack-mount servers. In a simple network where all your VM’s are on the same VLAN, it may make sense to do this on blades, too. Our deployment is more complex with many VLANs for our VM’s. In a blade configuration with 6 total NICs possible, we would be limiting our capabilities if we did not do the tagging. Hope this helps…

]]> By: M http://tech.philipsellers.com/2009/03/09/hp-blades-virtual-connect-esx-considerations/comment-page-1/#comment-508 M Tue, 10 Mar 2009 16:17:18 +0000 http://tech.philipsellers.com/?p=368#comment-508 Hya, We currently have the "Tunnel VLAN Tags" option enabled, and we have created networks that comprise of multiple Vlans. Still our ESX'es can talk to eachother of each Vlan, do the ESX'es use a different NiC per Vlan ? (The vSwitches have both nic's in them) greetz! M Hya,

We currently have the “Tunnel VLAN Tags” option enabled, and we have created networks that comprise of multiple Vlans. Still our ESX’es can talk to eachother of each Vlan, do the ESX’es use a different NiC per Vlan ? (The vSwitches have both nic’s in them)

greetz!

M

]]>