Qaucli Command Hangs When Trying to Upgrade a Qlogic CNA

Asset ID:	1-72-2156331.1
Update Date:	2018-02-08
Keywords:

Solution Type Problem Resolution Sure

Solution 2156331.1 : Qaucli Command Hangs When Trying to Upgrade a Qlogic CNA

Applies to:

SPARC T5-2 - Version All Versions and later
Information in this document applies to any platform.

Symptoms

One may experience the qaucli command hanging when trying to upgrade a Qlogic CNA firmware using the update_sol.sh script.

root@primary:~/QLE8362# sh -x update_sol.sh .
+ MBIDIR=.
+ OPTDIR=/opt/QLogic_Corporation/
+ [ 1 -eq 0 ]
+ find /opt/QLogic_Corporation/ -type f -name qaucli.cfg -print
+ tee
+ SIGFILE=/opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli.cfg
+ [ /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli.cfg '=' '' ]
+ dirname /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli.cfg
+ PROGPATH=/opt/QLogic_Corporation/QConvergeConsoleCLI
+ EXEBIN=/opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli
+ find . -type f -name OL862BCb.dat -print
+ DATFILE=./OL862BCb.dat
+ echo Flashing Board Config Data...
Flashing Board Config Data...
+ /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli -qmfgoem /hld all /cfl ./OL862BCb.dat
Using config file: /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli.cfg
Installation directory: /opt/QLogic_Corporation/QConvergeConsoleCLI
Working dir: /root/QLE8362

root@host1:~/QLE8362# ps -ef |grep qaucli |grep -v grep
root 909 904 0 11:10:34 pts/1 0:00 /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli -qmfgoem /hld all /cfl ./OL8

Cause

By trussing the script execution, we notice the qaucli binary is looping indefinitely in an ioctl on a qlogic device :

root@primary: truss --laefdD update_sol.sh

(...)

789/1: 0.3082 0.0000 open("/devices/pci@380/pci@1/pci@0/pci@5/SUNW,qlc@0:devctl", O_RDWR) = 3

(...)

789/1: 12.5985 0.0001 ioctl(3, 0x0069, 0xFFBFF2A8) = 0
789/1: 12.8961 0.2976 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 12.8972 0.0011 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 12.8984 0.0012 ioctl(3, 0x0069, 0xFFBFF268) = 0
789/1: 12.8985 0.0001 ioctl(3, 0x0069, 0xFFBFF2A8) = 0
789/1: 13.1964 0.2979 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 13.1975 0.0011 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 13.1987 0.0012 ioctl(3, 0x0069, 0xFFBFF268) = 0
789/1: 13.1988 0.0001 ioctl(3, 0x0069, 0xFFBFF2A8) = 0
789/1: 13.4960 0.2972 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 13.4972 0.0012 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 13.4983 0.0011 ioctl(3, 0x0069, 0xFFBFF268) = 0
789/1: 13.4985 0.0002 ioctl(3, 0x0069, 0xFFBFF2A8) = 0
789/1: 13.7960 0.2975 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 13.7972 0.0012 ioctl(3, 0x0069, 0xFFBFF2F0) = 0
789/1: 13.7983 0.0011 ioctl(3, 0x0069, 0xFFBFF268) = 0

(...)

That path is not known on the host primary :

root@primary: luxadm -e port
/devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:devctl CONNECTED
/devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0,1/fp@0,0:devctl CONNECTED

That is actually a HBA that has been assigned to the secondary I/O domain :

root@secondary: luxadm -e port
/devices/pci@380/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:devctl CONNECTED
/devices/pci@380/pci@1/pci@0/pci@5/SUNW,qlc@0,1/fp@0,0:devctl CONNECTED

As reported also by ldm :

root@primary@ ldm list -o physio

NAME
primary

IO
DEVICE PSEUDONYM OPTIONS
pci@340 pci_1
pci@300 pci_0
pci@340/pci@1/pci@0/pci@4 /SYS/MB/PCIE5
pci@340/pci@1/pci@0/pci@5 /SYS/MB/PCIE6
pci@340/pci@1/pci@0/pci@6 /SYS/MB/PCIE7
pci@300/pci@1/pci@0/pci@4 /SYS/MB/PCIE1
pci@300/pci@1/pci@0/pci@2 /SYS/MB/SASHBA0
pci@300/pci@1/pci@0/pci@1 /SYS/MB/NET0
------------------------------------------------------------------------------
NAME
secondary

IO
DEVICE PSEUDONYM OPTIONS
pci@3c0 pci_3
pci@380 pci_2
pci@3c0/pci@1/pci@0/pci@7 /SYS/MB/PCIE8
pci@3c0/pci@1/pci@0/pci@2 /SYS/MB/SASHBA1
pci@3c0/pci@1/pci@0/pci@1 /SYS/MB/NET2
pci@380/pci@1/pci@0/pci@5 /SYS/MB/PCIE2
pci@380/pci@1/pci@0/pci@6 /SYS/MB/PCIE3
pci@380/pci@1/pci@0/pci@7 /SYS/MB/PCIE4
------------------------------------------------------------------------------

However there is still links in /dev/cfg/ and /dev/fc on the primary I/O domain referring to that HBA :

root@primary: ls -l /dev/cfg

total 14
lrwxrwxrwx 1 root root 59 Jun 4 2015 c1 -> ../../devices/pci@3c0/pci@1/pci@0/pci@2/scsi@0/iport@8:scsi
lrwxrwxrwx 1 root root 60 Jun 4 2015 c10 -> ../../devices/pci@380/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:fc
lrwxrwxrwx 1 root root 62 Jun 4 2015 c11 -> ../../devices/pci@380/pci@1/pci@0/pci@5/SUNW,qlc@0,1/fp@0,0:fc
lrwxrwxrwx 1 root root 60 Jun 4 2015 c2 -> ../../devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:fc
lrwxrwxrwx 1 root root 62 Jun 4 2015 c3 -> ../../devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0,1/fp@0,0:fc
lrwxrwxrwx 1 root root 59 Jun 4 2015 c4 -> ../../devices/pci@300/pci@1/pci@0/pci@2/scsi@0/iport@1:scsi
lrwxrwxrwx 1 root root 59 Jun 4 2015 c5 -> ../../devices/pci@300/pci@1/pci@0/pci@2/scsi@0/iport@2:scsi
lrwxrwxrwx 1 root root 60 Jun 4 2015 c6 -> ../../devices/pci@300/pci@1/pci@0/pci@2/scsi@0/iport@v0:scsi
lrwxrwxrwx 1 root root 59 Jun 4 2015 c7 -> ../../devices/pci@3c0/pci@1/pci@0/pci@2/scsi@0/iport@1:scsi
lrwxrwxrwx 1 root root 59 Jun 4 2015 c8 -> ../../devices/pci@3c0/pci@1/pci@0/pci@2/scsi@0/iport@2:scsi
lrwxrwxrwx 1 root root 60 Jun 4 2015 c9 -> ../../devices/pci@3c0/pci@1/pci@0/pci@2/scsi@0/iport@v0:scsi
drwxr-xr-x 18 root root 18 Jun 4 2015 usb0

root@primary: ls -l /dev/fc

total 4
lrwxrwxrwx 1 root root 64 Jun 4 2015 fp0 -> ../../devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:devctl
lrwxrwxrwx 1 root root 66 Jun 4 2015 fp1 -> ../../devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0,1/fp@0,0:devctl
lrwxrwxrwx 1 root root 64 Jun 4 2015 fp2 -> ../../devices/pci@380/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:devctl
lrwxrwxrwx 1 root root 66 Jun 4 2015 fp3 -> ../../devices/pci@380/pci@1/pci@0/pci@5/SUNW,qlc@0,1/fp@0,0:devctl

Solution

The solution is to cleanup both the /dev/cfg and /dev/fc directories to get ride of these links that should not be here anymore.

Run devfsadm -Cv
Check in /dev/cfg and /dev/fc that links to the reassigned HBA have been removed.
If you still have links, then remove them with "rm".

Once /dev/cfg/ and /dev/fc have been cleaned up, the script update_sol.sh will execute and finish without any problem.

Attachments

This solution has no attachment