Sun Fire X4270/X4275 May Panic with "pcieb-4: PCI(-X) Express Fatal Error. (0x101)"

Asset ID:	1-72-1359338.1
Update Date:	2018-04-02
Keywords:

Solution Type Problem Resolution Sure

Solution 1359338.1 : Sun Fire X4270/X4275 May Panic with "pcieb-4: PCI(-X) Express Fatal Error. (0x101)"

Applies to:

Sun Netra X4270 Server - Version Not Applicable to Not Applicable [Release N/A]
Sun Fire X4275 Server - Version Not Applicable to Not Applicable [Release N/A]
Sun Fire X4270 Server - Version Not Applicable to Not Applicable [Release N/A]
Sun Fire X4270 M2 Server - Version Not Applicable to Not Applicable [Release N/A]
x86_64

Symptoms

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - Sun x86 Systems

The following describes the fingerprints to confirm this issue occurred.

The system will panic with either of the following panic strings:

pcieb-4: PCI(-X) Express Fatal Error. (0x101)
pcieb-5: PCI(-X) Express Fatal Error. (0x101)
pcieb-4: PCI(-X) Express Fatal Error. (0x103)
pcieb-5: PCI(-X) Express Fatal Error. (0x103)
pcie_pci-5: PCI(-X) Express Fatal Error
pcie_pci-4: PCI(-X) Express Fatal Error

"SUNW-MSG-ID: SUNOS-8000-0G" will be seen in /var/adm/messages and 'fmadm faulty', eg:

genunix: [ID 843051 kern.info] NOTICE: SUNW-MSG-ID: SUNOS-8000-0G, TYPE: Error, VER: 1, SEVERITY: Major
unix: [ID 836849 kern.notice]
^Mpanic[cpu9]/thread=fffffe8000a65c60:
genunix: [ID 647700 kern.notice] pcieb-4: PCI(-X) Express Fatal Error. (0x101)
unix: [ID 100000 kern.notice]
genunix: [ID 655072 kern.notice] fffffe8000a65bf0 pcieb:pcieb_intr_handler+1ea ()
genunix: [ID 655072 kern.notice] fffffe8000a65c40 unix:av_dispatch_autovect+78 ()
genunix: [ID 655072 kern.notice] fffffe8000a65c50 unix:intr_thread+5f ()
unix: [ID 100000 kern.notice]

Running 'fmdump -e' will show the following events similar to the following:

ereport.io.pci.nr            <-- No response
ereport.io.pci.nr            <-- No response
ereport.io.pci.nr    <-- No response
ereport.io.pci.fabric    <-- PCIe/PCI Fabric Error
ereport.io.pciex.rc.ce-msg   <-- PCI Express root complex received a correctable error message
ereport.io.pci.fabric        <-- PCIe/PCI Fabric Error
ereport.io.pciex.rc.mce-msg <-- PCI Express root complex received multiple correctable errors
ereport.io.pci.fabric        <-- PCIe/PCI Fabric Error
ereport.io.pci.fabric    <-- PCIe/PCI Fabric Error
ereport.io.pci.fabric        <-- PCIe/PCI Fabric Error
ereport.io.pciex.pl.re       <-- PCI Express physical layer Receiver Error
ereport.io.pciex.rc.ce-msg   <-- PCI Express root complex received a correctable error message

This issue can be seen with or without PCI cards installed on the PCI Riser on the panicing pcie device path (pcieb-4 or pcieb-5)

Cause

<BUG: 15714954> identified an incorrect settings within the Lynx IDT PCIe switch which can, in rare circumstances, lead to the switch reporting receiver underflow or overflow events. This causes the switch link to drop leading to a surprise link down event and a reset of the switch.

When this event happens the primary PCI Riser is unresponsive which is why the ereport.io.pci.nr and/or ereport.io.pciex.pl.re events are detected by FMA.

Solution

The issue is confirmed to be resolved in the following BIOS releases and above (Always download the latest available for your platform to address more issues and optimise error handling/diagnosability):

Sun Fire X4270
- <Patch: 15909999> - X4270 SW 2.6.2 - ILOM and BIOS
Sun Fire X4270 M2
- <Patch: 14825340> - X4270 M2 SW 1.7 - ILOM and BIOS
Sun Netra X4270
- <Patch: 16058865> - Netra X4270 SW 1.2.1 - ILOM and BIOS
Sun Fire X4275
- <Patch: 15910003> - X4275 SW 2.6.2 - ILOM and BIOS

As detailed in the "Applies To" Section above, this issue does not affect the M3 range of servers and platforms that are 1 Rack Unit (RU) in height.

Attachments

This solution has no attachment