How to Read PCI Negotiated Speed on Qlogic FC HBA

Asset ID:	1-71-2000056.1
Update Date:	2018-05-08
Keywords:

Solution Type Technical Instruction Sure

Solution 2000056.1 : How to Read PCI Negotiated Speed on Qlogic FC HBA

Applies to:

SPARC T5-4 - Version All Versions and later
Information in this document applies to any platform.

Goal

To read the pci bus negotiated speed of a PCI 8Gb Oracle Qlogic FC HBAs installed on a T5-4 server.

On the T4 server platform exist an internal utility called idtutil.sh , that reads this information from a PCI switch on the T4 server, but for the T5 platform this does not exist.

Reason for knowing the pci negotiated speed: There was some misprogrammed Emulex HBAs operating @Gen2 instead of Gen1 in a number of SPARC T4-4 systems. This resulted in these devices reporting an extremely high rate of CEs and device faults, reference:

Emulex HBA faulted following ereport.io.pciex.pl.re events when operating at PCIe Gen2 (Doc ID 1624749.1)

Emulex opted to implement a fix on a specifi c FC HBA that forced the upstream port to train @Gen1rather than th e endpoint , leading to the confusion with prtdiag output that showed a value of "5.0GTx4" = Gen2when it was negotiating at "2.5GTx8" = Gen1.

Reference PCI-E Bus Errors Reported Against QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 371-4325-02 (Doc ID 1614962.1) for how to update the preload table version 2.5.4 which forces the HBA to negotiate to PCI-E generation 1 speed 2.5GT.

Reference T5-4/T5-8 : PCIE fabric error panic or/and ereports on QLogic Fiber Channel HBA (Pallene) populated slots (Doc ID 1559361.1) to obtain QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 7076907 with preload table version 2.5.4 already applied.

prtdiag is limited in that it only reports the speed/width of functio ns within the card, this indicate the maximum speed it can negotiate on a PCI bus,but it does not read the pci negotiated speed (current running speed).

If a function within a device is reported as training @Gen1 then obviously it cannot be operating any faster than that no matter what speed/width the device upstream port is negotiating with the PCI switch.

Solution

1. On the explorer we can see FC HBA 8GB "SG-XPCIE2FC-QF8-Z " is PCIe Gen1 , see on prtdiag -v value "2.5GTx8"

bash-3.2$ more prtdiag-v.out
System Configuration: Oracle Corporation sun4v SPARC T5-4
Memory size: 32768 Megabytes

....

================================= IO Devices =================================
Slot + Bus Name + Model Speed
Status Type Path
------------------------------------------------------------------------------

/SYS/RCSA/PCIE5 PCIE SUNW,qlc-pciex1077,2532 QLE2562 2.5GTx8
/pci@340/pci@1/pci@0/pci@e/SUNW,qlc@0
/SYS/RCSA/PCIE5 PCIE SUNW,qlc-pciex1077,2532 QLE2562 2.5GTx8
/pci@340/pci@1/pci@0/pci@e/SUNW,qlc@0,1

2. We can also confirm and read the "PCIe Negotiated Speed" information
with the "QLogic QConvergeConsole GUI/CLI Management Tools"

This utility comes with Solaris 11.2 as part of Oracle Server Management,
but before Solaris 11.2, we had to manually install it.

This tool is available from the Qlogic download site:
https://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/

Also, it comes with Oracle HMP (Hardware Management Pack), but OHMP utility must be removed as it is not for customer use.

# /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli
Using config file: /opt/QLogic_Corporation/QConvergeConsoleCLI/qaucli.cfg
Installation directory: /opt/QLogic_Corporation/QConvergeConsoleCLI
Working dir: /opt/QLogic_Corporation/QConvergeConsoleCLI

       QConvergeConsole

       CLI - Version 1.1.0 (Build 9)

   Main Menu

   1: Adapter Information
   2: Adapter Configuration
   3: Adapter Updates
   4: Adapter Diagnostics
   5: Adapter Statistics
   6: Refresh
   7: Help
   8: Exit

       Please Enter Selection: 1

       QConvergeConsole

       CLI - Version 1.1.0 (Build 9)

   FC Adapter Information

   1: FC Adapter Information
   2: FC Port Information
   3: FC VPD Information
   4: FC Target/LUN Information

       (p or 0: Previous Menu; m or 98: Main Menu; ex or 99: Quit)
       Please Enter Selection: 2

       QConvergeConsole

       CLI - Version 1.1.0 (Build 9)

   Adapter Information

   HBA Model QLE2562 SN: 463916T+1411183247
     1: Port   1: WWPN: 21-00-00-24-FF-XX-XX-5A Online
     2: Port   2: WWPN: 21-00-00-24-FF-XX-XX-5B Link Down

       (p or 0: Previous Menu; m or 98: Main Menu; ex or 99: Quit)
       Please Enter Selection: 1
--------------------------------------------------------------------------------
Host Name                      : server01
HBA Instance                   : 0
OS Instance                    : 0
HBA Model                      : QLE2562-S
HBA Description                : QLE2562 SUN StorageTek Enterprise 8Gb PCI-Express HBA Dual Port
HBA ID                         : 0-QLE2562-S
HBA Alias                      :
HBA Port                       : 1
Port Alias                     :
Node Name                      : 20-00-00-24-FF-XX-XX-5A
Port Name                      : 21-00-00-24-FF-XX-XX-5A
Port ID                        : 01-DA-00
Serial Number                  : 463916T+1411183247
Driver Version                 : qlc-20140422-5.01
FCode Version                  : 2.03
Running Firmware Version       : 5.08.00
Flash BIOS Version             : 2.02
Flash FCode Version            : 2.03
Flash EFI Version              : 2.01
Flash Firmware Version         : 4.03.02
Actual Connection Mode         : Point to Point
Actual Data Rate               : 4 Gbps
Device ISP ID                  : ISP2532
PortType (Topology)            : NPort
Target Count                   : 2
PCI Bus Number                 : 8
PCI Device Number              : 0
PCI Function Number            : 0
PCI Device ID                  : 0x2532
Subsystem Device ID            : 0x0171
Subsystem Vendor ID            : 0x1077
PCIe Max Bus Width             : x8
PCIe Negotiated Width          : x8   <<<-----!!!!!
PCIe Max Bus Speed             : 2.5 Gbps
PCIe Negotiated Speed          : 2.5 Gbps <<<-----!!!!!
HBA Temperature (C)            : N/A
HBA Status                     : Online
--------------------------------------------------------------------------------

References

<NOTE:1559361.1> - T5-4/T5-8 : PCIE fabric error panic or/and ereports on QLogic Fiber Channel HBA (Pallene) populated slots
<NOTE:1614962.1> - PCI-E Bus Errors Reported Against QLogic 8 Gigabit/Second PCI-E Dual Fibre Channel Host Bus Adapter (HBA) Part Number 371-4325-02
<NOTE:1624749.1> - Emulex HBA faulted following ereport.io.pciex.pl.re events when operating at PCIe Gen2

Attachments

This solution has no attachment