After banging my head for nearly 3 days straight I finally got this HP MSA Array to talk to Xenserver 6.0.2.

What should’ve been a simple point click next, next install turned out to be not quite so straight forward and just goes to prove that bloody Murphy, he’ll get you every time.

So the premise was simple install Xenserver 6 onto 2 HP 385 Gen8s connected via a couple of Juniper ex4200s to an HP MSA p2000 g3 array. All the gear was on the Citrix HCL list with the minor caveat that the MSA Array was only certified under Xenserver 5.6,surely that was just and oversight and they haven’t gotten around to testing it or something yet?

Well that’s where Murphy comes in, a simple dismissive thought like that will come back to bite you. All the hardware setup went smooth last week and around Friday I installed Xenserver and configured the networking without too many hassles. Everything pings, I can get to all the consoles or web interfaces all sweet, I’ll just add an iSCSI SR for some shared storage and we can start creating some VMs, that where the fun began. Trying to do an iSCSI discovery returned the IQNs for the target correctly (after a lengthy pause), however trying to scan for a LUN produced a cryptic error message which after a quick Google gave no clues whatsoever.

I figured seeing as Xen is built on opensource and uses the open-scsi implmentation I’d drop to CLI and see what was going on.

iscsiadm -m discovery -t sendtargets -p 192.168.50.1 

192.168.50.2:3260,3 iqn.1986-03.com.hp:storage.p2000g3.122014e94d
192.168.50.4:3260,4 iqn.1986-03.com.hp:storage.p2000g3.122014e94d
192.168.50.3:3260,2 iqn.1986-03.com.hp:storage.p2000g3.122014e94d
192.168.50.1:3260,1 iqn.1986-03.com.hp:storage.p2000g3.122014e94d

So that’s promising I can discover the IQNs

iscsiadm -m node -T iqn.1986-03.com.hp:storage.p2000g3.122014e94d -p 192.168.50.1:3260 --login 

Logging in to [iface: default, target: iqn.1986-03.com.hp:storage.p2000g3.122014e94d, portal: 192.168.50.1,3260]
iscsiadm: Could not login to [iface: default, target: iqn.1986-03.com.hp:storage.p2000g3.122014e94d, portal: 192.168.50.1,3260]:
iscsiadm: initiator reported error (11 - iSCSI PDU timed out)

Hmm not so promising, and it was that little error that sent me on a many day journey, that Citrix Support couldn’t solve.

I tested all the obvious things, tried another server, windows 2003 and 2008 both worked fine. Tried another Linux distro, I grabbed openfiler which is very nice and yes it too just worked straight out of the box. I even ran it as a VM on local storage of the Xenserver install, so that pretty much proved the networking and setup and as openfiler runs open-iscsi it even proved that works.

Along the way if trying to solve the problem I discovered a Citrix XenCenter console blanking issue. Despite everything else in XenCenter working smoothly the console would just refuse to work and showed blank, not very handy for trying to install a new VM when you can’t access the console on it. Again hunting many forums, there was suggestions of various solutions such as VPNs and having the XenConsole machine on the same subnet as the XenServer, which got me to thinking it might be a firewall issue. So I checked the Citrix site for the ports it uses and saw it’s basically 80,443 and 5900, then I got to thinking if it uses web ports maybe it’s not liking the proxy server, so I disabled the proxy in XenCenter and BOOM console works!

After a lot of  changing configurations, testing various parts and re-installing almost everything under the sun, I finally solved the main problem. I was pouring over the Citrix site and found something very interesting buried away that mentioned switching from openVswitch back to bridged mode fro XenServer 6.  Ding went the little light; they implemented openvSwitch back in XS5.6 and it seems it’s the default. now in 6. I thought given it looked like some crazy packets and at this point anything was worth a shot so I switched to bridged mode with the command xe-switch-network-backend bridge on both Servers in the pool and rebooted them.

This time an iscsadm –login worked!! yay! So I jumped into Xencenter, tried creating a new SR and voila I can now see the LUNs! Strange though that a VM using the same network interfaces was able to connect to the Array fine, however the XenServer dom0 interfaces had problems connecting to it.

Either way I’m just happy to have finally cracked what should’ve been such a simple thing, now I can get on with what I should be doing; creating VMs and that planing of the P2V. Here’s fingers crossed that I don’t waste another 3 days on what should be a simple task.