vmware KB 2014323 PowerCLI to set qla2xxx option at cluster level

I’ve been facing very recently a naughty issue when I updated my vSphere 4.1u2 FCoE hosts to vSphere 5.0u1.

Right after the migration to vSphere 5, majors’ performance issues appeared with the underling storage. I was facing read/write latencies of several thousand milliseconds which lead to dead path detection and of course I/O problem within the VMs.

The servers where working perfectly with vSphere 4.1, so it has been decided to quickly rollback from 5.0u1 to 4.1u2.

After having restored a proper virtualisation service I opened SR at VMware who asked me to open an SR at the storage array vendor, who asked me to open a SR at the FC/FCoE switch vendor. 3 SR for 1 problem L

The servers are Dell PowerEdge R715 with QLogic 8150 CNA cards.

After several diag files uploads, patches at the array level and further researches one of the support team (not VMware’s) came with a very interesting KB from VMware website -> I/O activity pauses on virtual machines with QLogic 81xx series CNA cards on ESXi 5.0

This KB sounded good to my ears. The workaround mentioned in this KB has been tested on a single host with the desired result -> no more latency or dead path.

So it has been decided to apply the workaround on every FCoE hosts.

The KB gives you the way to apply the workaround with esxcli. If you have a vMA and not too many hosts it is fine, but I wanted to be use a more “industrial” way. So I used PowerCLI.

My script below is designed to be applied at a cluster level for hosts running vSphere 5, having vmhba model “ISP81xx-based 10 GbE FCoE to PCI Express CNA” and using the driver “qla2xxx”. It will only run on hosts in maintenance mode and will ask for a reboot after its completion.

The model name mentioned above is the one returned by the cmdlet “Get-VMHostHba” for Qlogic 8150/8152 CNA cards. If you are using other model impacted by this issue as described in the Qlogic website, maybe check if it is the same model name. I’ve set a print mode for these info in the script.

It helped me a lot.

Eric

Download-Script

Start-Script->

## options dedicated to Qlogic CNAs and vSphere 5 issue -> VMware kb: http://kb.vmware.com/kb/2014323
## build 1.00
## Eric Krejci
## ekrejci.wordpress.com
## Twitter – @ekrejci
## on error stop the script
$erroractionpreference=“Stop“
$Username=“DOMAIN\USERNAME“
$Password=read-host“Enter Password“-assecurestring

### declare your vcenter

$vCenterserver=“vcenter.fqdn“

### declare the name of the target DataCenter

$TargetDCName=“DC-NAME“

### declare the name of the target Cluster

$TargetClusterName=“Cluster-Name“

### create an empty array to store the modified hosts

$AppliedESXs= @()

### connect to the vCenter
Connect-VIServer-Server$vCenterserver-user$Username-Password ([Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($password)))

## retreive the vSphere in the target cluster with powerstate -> PoweredOn AND vSphere version equal or larger to 5

$TargetESXs=get-vmhost-Location (Get-Cluster-Name$TargetClusterName-Location (Get-Datacenter-Name$TargetDCName )) | where {($_.powerstate -eq“PoweredOn“) -and ($_.Version -ge“5“)}

foreach ($TargetESXin$TargetESXs ) {

$TargetESX.name

## we check if the host is in maintenance mode to avoid any risks when performing the actions. if you don’t want to set the maintenance mode, the script ends.

if ($TargetESX.ConnectionState-eq“Maintenance“) {
$TargetESX.name+“ is already in maintenance mode“
} else {
if (“Y“-eq ((Read-Host“Your ESX is not in maintenance! Do you want to set now? Enter Y or N“).ToUpper())) {

Set-VMHost-VMHost$TargetESX-StateMaintenance

} else {

“exit“
exit

}
}

## uncomment the following to print the name, driver and model of the current vSphere’s VMHBAs

#Get-VMHostHba -VMHost $TargetESX | %{“Name: ” + $_.name + “; driver: ” + $_.driver + “; Model: ” +$_.model}

## retreive every VMHBA of the server with model “*10 GbE FCoE to PCI Express CNA*” and using the driver “qla2xxx”

$TargetESXHBAs=Get-VMHostHba-VMHost$TargetESX | where {($_.driver -eq“qla2xxx“) -and ($_.model -like“*10 GbE FCoE to PCI Express CNA*“) }

## if we have corresponding Qlogic CNAs then we apply the option described in the VMware KB.

if ($TargetESXHBAs-ne$null ) {

## getting the module “qla2xxx” from the current vSphere

$qla2xxx=Get-VMHostModule-VMHost$TargetESX-Name“qla2xxx“

## setting the module “qla2xxx” with the good option

$qla2xxx | Set-VMHostModule-Options“ql2xenablemsix=0“

$AppliedESXs+=$TargetESX

}
}

## you must reboot to have the setting applied. if you don’t want to do it now, don’t forget to reboot the hosts later.

if (“Y“-eq ((Read-Host“Do you want to restart the ESX Servers? Enter Y or N“).ToUpper())) {
“Rebooting the ESX Server“

foreach ($AppliedESXin$AppliedESXs ) {

$AppliedESXView=Get-View-Id$AppliedESX.id

$AppliedESXView.RebootHost($false)

“End of configure ESX server “+$AppliedESX

}
}

“Finished“

Disconnect-VIServer-Confirm:$false

<-End-Script