How To Set Qlogic CNA Personality - Sun Storage Qlogic 16Gb FC HBA Universal / 10GB FCoE - Report SFP: Invalid, Reason: 1000 at the OBP

Asset ID:	1-72-1924039.1
Update Date:	2018-04-03
Keywords:

Solution Type Problem Resolution Sure

Solution 1924039.1 : How To Set Qlogic CNA Personality - Sun Storage Qlogic 16Gb FC HBA Universal / 10GB FCoE - Report SFP: Invalid, Reason: 1000 at the OBP

Applies to:

SPARC T4-4 - Version Not Applicable and later
SPARC T5-4 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

NOTE: This document only applies to Oracle 16Gb FC Universal (FC and CNA) HBA cards not to Oracle CNA only cards.

While trying to setup a SunStorage 16Gb Fibre Channel PCIe Universal Host Bus Adapter, QLogic in the FC protocol mode by installing 16 Gb FC shortwave optical transceiver modules in the SFP+ connector of the universal HBA, the link may not come up with both the following symptom.

NOTE: This issue can also occur when all parameters are set properly but NON-Oracle SFPs are used. Only Oracle SFPs are supported.

When selecting the device at the OBP, you will get the undocumented error code :

{0} ok show-disks
a) /pci@400/pci@2/pci@0/pci@f/pci@0/usb@0,2/hub@2/hub@3/storage@2/disk
b) /pci@400/pci@2/pci@0/pci@a/SUNW,qlc@0,3/fp@0,0/disk
c) /pci@400/pci@2/pci@0/pci@a/SUNW,qlc@0,2/fp@0,0/disk
d) /pci@400/pci@2/pci@0/pci@4/scsi@0/disk
e) /pci@400/pci@1/pci@0/pci@4/scsi@0/disk
f) /iscsi-hba/disk
q) NO SELECTION
Enter Selection, q to quit: q
{0} ok select /pci@400/pci@2/pci@0/pci@a/SUNW,qlc@0,3
QLogic QLE8362 FCoE Host Adapter FCode(SPARC): 4.02 2012-11-12 13:52
ISP Firmware version 6.06.03

NIC FW version 5.00.41

Fibre Channel Link down
Current firmware state - Loss of Sync - Reason Code: 0
SFP: Invalid, Reason: 1000 -
{0} ok

After booting, you will get the following error in messages :

Mar 16 10:41:08 server01 qlc: [ID 139792 kern.info] Qlogic qlc(6) FCA Driver v150505-5.03b
Mar 16 10:41:09 server01 qlc: [ID 279254 kern.info] Qlogic qlc(6,0,2) WWPN=2100000e1XXXXXX0 : WWNN=2000000e1XXXXXX0
Mar 16 10:41:09 server01 qlc: [ID 700582 kern.info] NOTICE: Qlogic qlc(6,0,2): Invalid SFP, reason=1000h <<<----!!!!!
Mar 16 10:41:18 server01 qlc: [ID 630585 kern.info] NOTICE: Qlogic qlc(6): Loop OFFLINE
Mar 16 10:41:18 server01 qlc: [ID 297195 kern.info] NOTICE: Qlogic qlc(6): BIOS: 3.19; fcode: 4.02; EFI: 5.36;
Mar 16 10:41:18 server01 qlc: [ID 874193 kern.info] NOTICE: Qlogic qlc(6): Firmware version 7.05.00, nic v5.0.41, phy v0.0.0, device id 8031
Mar 16 10:41:18 server01 pcieb: [ID 586369 kern.info] PCIE-device: SUNW,qlc@0,2, qlc6

Mar 16 10:41:18 server01 qlc: [ID 139792 kern.info] Qlogic qlc(7) FCA Driver v150505-5.03b
Mar 16 10:41:18 server01 qlc: [ID 279254 kern.info] Qlogic qlc(7,0,3) WWPN=2100000e1XXXXXX1 : WWNN=2000000e1XXXXXX1
Mar 16 10:41:18 server01 qlc: [ID 700582 kern.info] NOTICE: Qlogic qlc(7,0,3): Invalid SFP, reason=1000h <<<----!!!!!
Mar 16 10:41:28 server01 qlc: [ID 630585 kern.info] NOTICE: Qlogic qlc(7): Loop OFFLINE
Mar 16 10:41:28 server01 qlc: [ID 297195 kern.info] NOTICE: Qlogic qlc(7): BIOS: 3.19; fcode: 4.02; EFI: 5.36;
Mar 16 10:41:28 server01 qlc: [ID 874193 kern.info] NOTICE: Qlogic qlc(7): Firmware version 7.05.00, nic v5.0.41, phy v0.0.0, device id 8031
Mar 16 10:41:28 server01 pcieb: [ID 586369 kern.info] PCIE-device: SUNW,qlc@0,3, qlc7

Additionally, in Solaris "fcinfo hba-port" reports a supported speed at 10 Gb, when supported speed for FC are either 4, 8 or 16 Gb :

# fcinfo hba-port

root@t4-1-tvp540-a:~# fcinfo hba-port
HBA Port WWN: 2100000e1e125820
        Port Mode: Initiator
        Port ID: 0
        OS Device Name: /dev/cfg/c15
        Manufacturer: QLogic Corp.
        Model: 7023303
        Firmware Version: 6.06.03
        FCode/BIOS Version: BIOS: 3.19; fcode: 4.02; EFI: 5.36;
        Serial Number: 463916A+1315135436
        Driver Name: qlc
        Driver Version: 20140211-5.00
        Type: unknown
        State: offline
        Supported Speeds: 10Gb
        Current Speed: not established
        Node WWN: 2000000e1e125820
        Max NPIV Ports: 254
        NPIV port list:
HBA Port WWN: 2100000e1e125821
        Port Mode: Initiator
        Port ID: 0
        OS Device Name: /dev/cfg/c16
        Manufacturer: QLogic Corp.
        Model: 7023303
        Firmware Version: 6.06.03
        FCode/BIOS Version: BIOS: 3.19; fcode: 4.02; EFI: 5.36;
        Serial Number: 463916A+1315135436
        Driver Name: qlc
        Driver Version: 20140211-5.00
        Type: unknown
        State: offline
        Supported Speeds: 10Gb
        Current Speed: not established
        Node WWN: 2000000e1e125821
        Max NPIV Ports: 254
        NPIV port list:

Cause

There is a simple explanation to these symptoms :

The personality is currently set to CNA ( FCOE ) instead of FC :

{0} ok set-personality
Current Personality : CNA
Do you want to change it? (y/n) y
Choose Personality :
0 - FC only
1 - CNA
enter: 0
Current Personality : FC only
A PCIe-Reset is required for any change to take affect
Be sure to change the SFP modules

Solution

By changing the personality from FC to CNA, you will then get what you expect from both at the OBP and Solaris fcinfo :

The OBP will now report the SFP as present :

{0} ok show-disks
a) /pci@400/pci@2/pci@0/pci@f/pci@0/usb@0,2/hub@2/hub@3/storage@2/disk
b) /pci@400/pci@2/pci@0/pci@a/SUNW,qlc@0,1/fp@0,0/disk
c) /pci@400/pci@2/pci@0/pci@a/SUNW,qlc@0/fp@0,0/disk
d) /pci@400/pci@2/pci@0/pci@4/scsi@0/disk
e) /pci@400/pci@1/pci@0/pci@4/scsi@0/disk
f) /iscsi-hba/disk
q) NO SELECTION
Enter Selection, q to quit: q
{0} ok select /pci@400/pci@2/pci@0/pci@a/SUNW,qlc@0
QLogic QLE8362 FC Host Adapter FCode(SPARC): 4.02 2012-11-12 13:52
ISP Firmware version 6.06.03

Fibre Channel Link down
Current firmware state - Loss of Sync
SFP: Present

"fcinfo hba-port" will report the expected supported speed for the FC protocol :

root@t4-1-tvp540-a:~# fcinfo hba-port
HBA Port WWN: 2100000e1e125820
        Port Mode: Initiator
        Port ID: 0
        OS Device Name: /dev/cfg/c13
        Manufacturer: QLogic Corp.
        Model: 7023303
        Firmware Version: 6.06.03
        FCode/BIOS Version: BIOS: 3.19; fcode: 4.02; EFI: 5.36;
        Serial Number: 463916A+1315135436
        Driver Name: qlc
        Driver Version: 20140211-5.00
        Type: unknown
        State: offline
        Supported Speeds: 4Gb 8Gb 16Gb
        Current Speed: not established
        Node WWN: 2000000e1e125820
        Max NPIV Ports: 254
        NPIV port list:
HBA Port WWN: 2100000e1e125821
        Port Mode: Initiator
        Port ID: 0
        OS Device Name: /dev/cfg/c14
        Manufacturer: QLogic Corp.
        Model: 7023303
        Firmware Version: 6.06.03
        FCode/BIOS Version: BIOS: 3.19; fcode: 4.02; EFI: 5.36;
        Serial Number: 463916A+1315135436
        Driver Name: qlc
        Driver Version: 20140211-5.00
        Type: unknown
        State: offline
        Supported Speeds: 4Gb 8Gb 16Gb
        Current Speed: not established
        Node WWN: 2000000e1e125821
        Max NPIV Ports: 254
        NPIV port list:

Note. This is a Dual port card, the personality affect both ports, you cannot set just ONE port to CNA and the other port to FC.

Instructions to change the personality of this card to FC HBA can be found on the installation guide for the SunStorage 16Gb Fibre Channel PCIe Universal Host Bus Adapter, QLogic is available here

You can use QConvergeConsole CLI that comes with Solaris 11.2 on:
/usr/lib/ssm/fwupdate/qlogic/qaucli

See page 42 "Determining and Changing the Operating Protocol Mode Using the QConvergeConsole CLI"
of the "Sun Storage 16 Gb Fibre Channel PCIe Universal Host Bus Adapter, QLogic Installation Guide"

Once personality of card has been changed, it is required to power off/on the FC HBA,
as explained on page 43 of the "Sun Storage 16 Gb Fibre Channel PCIe Universal Host Bus Adapter, QLogic Installation Guide"

"then perform a full power reboot of the system in which the universal HBAs are installed.

Note - For systems that have hot-plug/hot swap capability, instead of initiating a full power
reboot of the system, you can perform a hot-plug out and hot-plug in of the card."

Examples using the CLI

To help identify PCI devices:

Fibre Channel PCI Identifiers are Vendor ID 0x1077 and Device ID 0x2031.
Ethernet PCI Identifiers are Vendor ID 0x1077 and Device ID 0x8030.
FCoE PCI Identifiers are Vendor ID 0x1077 and Device ID 0x8031.

/SYS/RCSA/PCIE3 PCIE SUNW,qlc-pciex1077,2031 QLE8362 8.0GT/x8 8.0GT/x4 <<<<<<<<<< Fibre Channel PCI
/pci@340/pci@1/pci@0/pci@6/SUNW,qlc@0
/SYS/RCSA/PCIE3 PCIE SUNW,qlc-pciex1077,2031 QLE8362 8.0GT/x8 8.0GT/x4 <<<<<<<<<< Fibre Channel PCI
/pci@340/pci@1/pci@0/pci@6/SUNW,qlc@0,1

/SYS/RCSA/PCIE10 PCIE SUNW,qlc-pciex1077,8031 QLE8362 8.0GT/x8 8.0GT/x4 <<<<<<<<<<< FCoE PCI / CNA PCI
/pci@380/pci@1/pci@0/pci@4/SUNW,qlc@0,2
/SYS/RCSA/PCIE10 PCIE SUNW,qlc-pciex1077,8031 QLE8362 8.0GT/x8 8.0GT/x4
/pci@380/pci@1/pci@0/pci@4/SUNW,qlc@0,3

To check / modify the adapter personality:

- Verify the operating protocol mode (personality) of all the universal HBAs installed with the following command:

# /usr/lib/ssm/fwupdate/qlogic/qaucli -pc all --info

- Change the operating protocol mode (personality) of all the universal HBAs installed to Fibre Channel only with the following command:

# /usr/lib/ssm/fwupdate/qlogic/qaucli -pc all --type fco

- Change the operating protocol mode (personality) of all the universal HBAs installed to CNA with the following command:

# /usr/lib/ssm/fwupdate/qlogic/qaucli -pc all --type cna

- Verify the SFP type installed with the following command:

# /usr/lib/ssm/fwupdate/qlogic/qaucli -dm all general

Attachments

This solution has no attachment