Unable To Create SR-IOV Virtual Instance For Qlogic 16 Gb Fibre Channel PCIe Universal Host Bus Adapter

Asset ID:	1-72-1950167.1
Update Date:	2018-02-23
Keywords:

Solution Type Problem Resolution Sure

Solution 1950167.1 : Unable To Create SR-IOV Virtual Instance For Qlogic 16 Gb Fibre Channel PCIe Universal Host Bus Adapter

Applies to:

SPARC M6-32 - Version All Versions and later
Qlogic FC HBA - Version Not Applicable to Not Applicable [Release N/A]
Information in this document applies to any platform.

Symptoms

Unable to create SR-IOV virtual function for QLogic 16 Gb Fibre Channel PCIe Universal Host Bus Adapter.

Issue applies to both the following parts:

Xoption Marketing part number 7101674
Card Assembly/FRU/CRU part number 7023303

Xoption Marketing part number 7101682
Card Assembly/FRU/CRU part number 7023304

http://twiki.us.oracle.com/bin/view/Main/HbaIdentification

No physical functions are listed for the QLogic HBA.
These are a requirement in order to create the virtual function.

For example:

# ldm ls-io

NAME TYPE BUS DOMAIN STATUS
---- ---- --- ------ ------
pci_0 BUS pci_0 primary IOV
pci_1 BUS pci_1 primary IOV
niu_0 NIU niu_0 primary
niu_1 NIU niu_1 primary
/SYS/MB/PCIE0 PCIE pci_0 primary OCC
/SYS/MB/PCIE2 PCIE pci_0 primary OCC
/SYS/MB/PCIE4 PCIE pci_0 primary EMP
/SYS/MB/PCIE6 PCIE pci_0 primary EMP
/SYS/MB/PCIE8 PCIE pci_0 primary EMP
/SYS/MB/SASHBA PCIE pci_0 primary OCC
/SYS/MB/NET0 PCIE pci_0 primary OCC
/SYS/MB/PCIE1 PCIE pci_1 primary EMP
/SYS/MB/PCIE3 PCIE pci_1 primary OCC
/SYS/MB/PCIE5 PCIE pci_1 primary OCC
/SYS/MB/PCIE7 PCIE pci_1 primary OCC
/SYS/MB/PCIE9 PCIE pci_1 primary OCC
/SYS/MB/NET2 PCIE pci_1 primary OCC
/SYS/MB/NET0/IOVNET.PF0 PF pci_0 primary
/SYS/MB/NET0/IOVNET.PF1 PF pci_0 primary
/SYS/MB/PCIE5/IOVNET.PF0 PF pci_1 primary
/SYS/MB/PCIE5/IOVNET.PF1 PF pci_1 primary
/SYS/MB/PCIE7/IOVFC.PF0 PF pci_1 primary <<<---- Fibre Channel Physical Functions will not be listed
/SYS/MB/PCIE7/IOVFC.PF1 PF pci_1 primary <<<---- Fibre Channel Physical Functions will not be listed
/SYS/MB/NET2/IOVNET.PF0 PF pci_1 primary
/SYS/MB/NET2/IOVNET.PF1 PF pci_1 primary

Changes

Cause

Regardless of the firmware running on the HBA a single one time operation is required to enable FC SR-IOV on Oracle QLogic Fibre Channel and Converged Network Adapters.

In this case the customer was very much below the required version of Firmware (07.03) on the HBA.

# fcinfo hba-port
HBA Port WWN: 2100000xxxxxxxxx
       Port Mode: Initiator
       Port ID: 37b00
       OS Device Name: /dev/cfg/c36
       Manufacturer: QLogic Corp.
       Model: 7023303
       Firmware Version: 6.06.03 <<<---- Below required version to support SR-IOV
       FCode/BIOS Version: BIOS: 3.27; fcode: 4.09; EFI: 6.10;
       Serial Number: 463916T+1418187761
       Driver Name: qlc
       Driver Version: 20140422-5.01
       Type: N-port
       State: online
       Supported Speeds: 4Gb 8Gb 16Gb
       Current Speed: 8Gb
       Node WWN: 2000000xxxxxxxxx
       Max NPIV Ports: 254
       NPIV port list:

However, even if the card is running a later version of firmware (shipped with more recent stock) the SR-IOV feature will not be available until QLogic's Multi-Boot Image Update Kit has been applied.

See doc
<Document 1325454.1> Oracle VM Server for SPARC PCIe Direct I/O and SR-IOV Features
search for Xoption Marketing part number

# fcinfo hba-port
HBA Port WWN: 2100000xxxxxxxxx
       Port Mode: Initiator
       Port ID: d1000
       OS Device Name: /dev/cfg/c5
       Manufacturer: QLogic Corp.
       Model: 7023303
       Firmware Version: 7.05.01 <<<---- Latest (at time of publishing) Firmware but same issue. No PF instances listed.
       FCode/BIOS Version: BIOS: 3.19; fcode: 4.02; EFI: 5.36;
       Serial Number: 463916T+1428205172
       Driver Name: qlc
       Driver Version: 150505-5.03b
       Type: N-port
       State: online
       Supported Speeds: 4Gb 8Gb 16Gb
       Current Speed: 8Gb
       Node WWN: 2000000xxxxxxxxx
       Max NPIV Ports: 254
       NPIV port list:

Firmware Preload table version 3.10.0 or later is required to support physical function creation.

Use the following QConvergeConsoleCLI command to determine the firmware preload table version, where the PWWN of the HBA can be determined from 'fcinfo hba-port':

# qaucli -u 2100000e1e120c80 /version
Using config file: /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli.cfg
Installation directory: /opt/QLogic_Corporation/QConvergeConsoleCLI
Working dir: /home/llmoss
Current firmware preload table version of HBA instance 12 (QEM8362 ) is v3.10.0

Alternatively, if the Operating System is 11.2 or later:

# /usr/lib/ssm/fwupdate/qlogic/qaucli -u 2100000e1e120c80 /version
Installation directory: /usr/lib/ssm/fwupdate/qlogic
Working dir: /home/llmoss
Current firmware preload table version of HBA instance 12 (QEM8362 ) is v3.10.0

Please note, it is recommended to install QConvergeConsoleCLI if the Preload firmware table needs updating. See the Solution (Additional Information) notes for more detail.

QConvergeConsoleCLI is available from the QLogic website in the Management Tools for Solaris section.
http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/SearchByProductOracle.aspx?oemid=124&productid=1146&OSTYPE=Solaris&category=11&Product=7101674

Solution

A one time manual action using a special script (Multi-Boot Image Update Kit to enable FC SR-IOV on Oracle Fibre Channel and Converged Network Adapters) is required.
See http://docs.oracle.com/cd/E24651_01/html/E24460/z40007c01004751.html

1. Navigate to http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/Oracle_Search.aspx
2. Select the following:
Converged Network Adapters
7101674 (in this example - select 7101682 if you are using the ExpressModule version)
Solaris
3. Enter the "Go" button
4. Download the "Multi-Boot Image Update Kit for Universal QLE8362 (7101674) HBA" and follow the instructions under the SR-IOV MBI Kit Read Me link

If the update kit you require is unavailable from the QLogic site then please raise a Service Request with Oracle Support against the HBA stating that you cannot download the Multi-Boot Image Update Kit for QLogic HBA. Also quote this Knowledge Document.

Please note.

This Multi-Boot Image Update Kit contains FC Firmware 7.03.02. If the HBA in question has a later release already installed then this firmware is not downgraded as part of the update.

Additional Information:

For versions of Solaris at 11.2 and later the qaucli utility is already available in /usr/lib/ssm/fwupdate/qlogic and so QConvergeConsoleCLI may not have been installed. If this is the case the update_sol.sh script will fail because it only checks for the existence of QConvergeConsoleCLI.

It will fail with the following response:

# ./update_sol.sh .
find: stat() error /opt/QLogic_Corporation/: No such file or directory
Please install the flash update utility and try again!

To resolve this either:

1. Edit the update_sol.sh script in the SRIOV_Flash_Kit to also check for qaucli in /usr/lib/ssm/fwupdate/qlogic (See the internal note for more information regarding this *** )

2. Install QConvergeConsoleCLI

This utility is available from the QLogic website in the Management Tools for Solaris section on the same page as the Multi-Boot Image Update Kit.

Example:

# pkgadd -d QConvergeConsoleCLI-1.1.03-38-sol8-sol12-sparc.pkg

The following packages are available:
1 QCCLI QLogic QConvergeConsole CLI (CNA Configuration Utility)
(sparc) 1.01.03 Build 38 (Solaris 8-12)

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]: 1

Processing package instance <QCCLI> from </home/cbeg/QConvergeConsoleCLI-1.1.03-38-sol8-sol12-sparc.pkg>

QLogic QConvergeConsole CLI (CNA Configuration Utility)(sparc) 1.01.03 Build 38 (Solaris 8-12)
QLogic Corporation
## Executing checkinstall script.
## Processing package information.
## Processing system information.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user
permission during the process of installing this package.

Do you want to continue with the installation of <QCCLI> [y,n,?] y

Installing QLogic QConvergeConsole CLI (CNA Configuration Utility) as <QCCLI>

## Installing part 1 of 1.
/opt/QLogic_Corporation/QConvergeConsoleCLI/adapters.properties
/opt/QLogic_Corporation/QConvergeConsoleCLI/agent.properties
/opt/QLogic_Corporation/QConvergeConsoleCLI/allpr_contents.dat
/opt/QLogic_Corporation/QConvergeConsoleCLI/fc_contents.dat

...

WARNING

Historically, these links have at times been broken on the QLogic website, particularly the QLE version which has pointed to the wrong file. This should be resolved now but in the event you or the customer have problems downloading the correct kit then they can be retrieved from the following source.

Option 7101674 - /home/tepp/qlc/Oracle_QLE8362_SRIOV_Flash_Kit_06.zip
Option 7101682 - /home/tepp/qlc/Oracle_QEM8362_SRIOV_Flash_Kit_06.zip

*** Tim Epp has edited the update_sol.sh scripts in both the above kits to check for qaucli in /usr/lib/ssm/fwupdate/qlogic as well as /opt/QLogic_Corporation/QConvergeConsoleCLI (the default location of qaucli after installing QConvergeConsoleCLI).

Oracle_QLE8362_SRIOV_Flash_Kit_07.zip

References

<NOTE:1325454.1> - Oracle VM Server for SPARC PCIe Direct I/O and SR-IOV Features
<NOTE:1672221.1> - Oracle Solaris 11.2 Support Repository Updates (SRU) Index
<NOTE:1935095.1> - Oracle Solaris 11.2.3.4.1 Support Repository Update (Oracle Solaris 11.2 SRU3.4.1) README
<NOTE:1514218.1> - Solaris[TM] 11 (not for Solaris[TM] 10 or before) Oracle Fibre Channel (FC) HBA Driver and Firmware SAN Patch Matrix
<BUG:21778430> - PHYSICAL FUNCTION MISSING FOR FC QLOGIC QLE8362 CARDS

Attachments

This solution has no attachment