PCI-E Bus Errors Reported Against QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 371-4325-02

Asset ID:	1-71-1614962.1
Update Date:	2018-05-08
Keywords:

Solution Type Technical Instruction Sure

Solution 1614962.1 : PCI-E Bus Errors Reported Against QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 371-4325-02

Applies to:

SPARC T5-8 - Version All Versions and later
SPARC T5-4 - Version All Versions and later
SPARC T5-2 - Version All Versions and later
SPARC T4-1 - Version All Versions and later
SPARC T3-1 - Version All Versions and later
Oracle Solaris on SPARC (64-bit)

Goal

PCI-E bus errors were encountered during testing of the QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 371-4325-02 with SPARC T5-4 / SPARC T5-8 servers. Firmware preload table version 2.5.4 was created to force the HBA to negotiate to PCI-E generation 1 speed 2.5GT versus the PCI-E generation 2 speed 5.0GT in order to work around the compatibility issue.

Rather than make firmware preload table version 2.5.4 available as download, which was how similar issues were handled with prior platforms, the SPARC T5 platform team chose to only have the firmware preload table updated on the HBA in the factory under a new part number 7076907. This decision unnecessarily prevents any part number 371-4325-02 HBAs in customer stock for use in SPARC T5-4 / SPARC T5-8 servers.

The procedure below can be used to update the QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 371-4325-02 firmware preload table to version 2.5.4 in order to force the HBA to negotiate to PCI-E generation 1 speed 2.5GT and make the HBA compatible with SPARC T5-4 / SPARC T5-8 servers.

Preload table version 2.5.4 can be applied to QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 371-4325-02 installed in any server platform which has negotiated to PCI-E generation 2 speed 5.0GT and encounters PCI-E bus related errors similar to the following:

fault.io.pciex.device-pcie-ce
fault.io.pciex.bus-linkerr-corr

Solution

- Download the QLogic QConvergeConsole CLI from the Oracle OEM QLogic website.

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/SearchByProductOracle.aspx?oemid=124&productid=928&OSTYPE=Solaris&category=7&Product=SG-XPCIE2FC-QF8-Z

Note for Solaris 11.2 and later, the QLogic QConvergeConsole CLI is already installed in /usr/lib/ssm/fwupdate/qlogic/qaucli.
Reference Solaris 11.2 (and later) FC HBA - Update Firmware, FCode/BIOS (ie. Boot Code) (Doc ID 1955822.1)

Please take the this opportunity to update the Multi-boot image for 8Gb FC HBAs if available as well.

- Download Firmware Preload Table Sun2562F.dat.

- Install QLogic QConvergeConsole CLI (Solaris 11 example):

% sudo pkg list -fav -g file:///var/tmp/QConvergeConsoleCLI-1.1.00-24-solaris-sparc.p5p
FMRI                                                                         IFO
pkg://qlogic/qlogic/qaucli@1.1,0-24:20130613T233310Z                         ---

% sudo pkg install -g file:///var/tmp/QConvergeConsoleCLI-1.1.00-24-solaris-sparc.p5p pkg://qlogic/qlogic/qaucli
           Packages to install: 1
       Create boot environment: No
Create backup boot environment: No
            Services to change: 1

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                                1/1       288/288      5.0/5.0 11.5M/s

PHASE                                          ITEMS
Installing new actions                       506/506
Updating package state database                 Done
Updating image state                            Done
Creating fast lookup database                   Done

- Start QLogic QConvergeConsole CLI:

% sudo qaucli

        QConvergeConsole

        CLI - Version 1.1.0 (Build 24)

    Main Menu

    1: Adapter Information
    2: Adapter Configuration
    3: Adapter Updates
    4: Adapter Diagnostics
    5: Adapter Statistics
    6: Refresh
    7: Help
    8: Exit

- Select Adapter Updates:

        Please Enter Selection: 3

        QConvergeConsole

        CLI - Version 1.1.0 (Build 24)

    Fibre Channel Adapter Update

    1: Flash Update
    2: Parameters Update
    3: Parameters Template Update
    4: Firmware Preload Update
    5: FC Serdes Table Update
    6: FC Dump

- Select Firmware Preload Update:

        Please Enter Selection: 4

        QConvergeConsole

        CLI - Version 1.1.0 (Build 24)

    Firmware Preload Update

- Select an adapter:

        Please Enter Selection: 1

        Firmware Preload Table
        Current version: v2.5.2

- Enter file name of the firmware preload table:

Enter a file name or Hit <RETURN> to abort: Sun2562F.dat

- Proceed with update.

- Repeat for each adapter.

- Exit QLogic QConvergeConsole CLI:

(p or 0: Previous Menu; m or 98: Main Menu; ex or 99: Quit)
Please Enter Selection: ex

- Another method is to utilize the command line option for the QConvergeConsole CLI Firmware Preload Table Update/Save USAGE:

Update the HBA firmware preload table from a DAT file.
qaucli -u <hba instance>|<hba wwpn>|<all> <file name>

Save the HBA firmware preload table to a DAT file.
qaucli -u <hba instance>|<hba wwpn> save <file name>

Display the HBA current firmware preload table version.
qaucli -u <hba instance>|<hba wwpn> /version

Options:
     <hba instance> The HBA instance number of an HBA port.
     <hba wwpn> The World Wide Port Name of an HBA port.
     <file name> Name of the firmware preload table DAT file.

Note: This feature is supported only with 8Gb/16Gb HBAs.

- Display the current firmware preload table version.

# /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli -u 0 /version

- Update the firmware preload table version on a specific qlc instance if required.

# /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli -u 0 ./Sun2562F.dat

- Or update the firmware preload table version on all the applicable qlc HBAs.

# /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli -u all ./Sun2562F.dat

- Hot plug the HBA or reboot the server. A power cycle is preferred and may be required.

- Verify the HBA has negotiated to PCI-E generation 1 speed 2.5GT:

% sudo prtdiag | grep QL
/SYS/RCSA/PCIE1 PCIE SUNW,qlc-pciex1077,2532 QLE2562 2.5GT/x8 2.5GTx8
/SYS/RCSA/PCIE1 PCIE SUNW,qlc-pciex1077,2532 QLE2562 2.5GT/x8 2.5GTx8
/SYS/RCSA/PCIE2 PCIE SUNW,qlc-pciex1077,2532 QLE2562    2.5GT/x8 2.5GTx8
/SYS/RCSA/PCIE2 PCIE SUNW,qlc-pciex1077,2532 QLE2562    2.5GT/x8 2.5GTx8
/SYS/RCSA/PCIE6 PCIE SUNW,qlc-pciex1077,2532 QLE2562    2.5GT/x8 2.5GTx8
/SYS/RCSA/PCIE6 PCIE SUNW,qlc-pciex1077,2532 QLE2562    2.5GT/x8 2.5GTx8
/SYS/RCSA/PCIE8 PCIE SUNW,qlc-pciex1077,2532 QLE2562    2.5GT/x8 2.5GTx8
/SYS/RCSA/PCIE8 PCIE SUNW,qlc-pciex1077,2532 QLE2562    2.5GT/x8 2.5GTx8

References

<NOTE:1389639.1> - FAQ Oracle FC HBA: FCode/BIOS(ie. Boot Code), Firmware, and Drivers
<NOTE:1559361.1> - T5-4/T5-8 : PCIE fabric error panic or/and ereports on QLogic Fiber Channel HBA (Pallene) populated slots
<NOTE:1955822.1> - Solaris 11.2 (and later) FC HBA - Update Firmware, FCode/BIOS (ie. Boot Code)

Attachments

This solution has no attachment