SPARC T5 System :FMA reports SUNOS-8000-FU or SUNOS-8000-J0 due to ereport.io.pciex.rc.ce-msg

Asset ID:	1-72-1612672.1
Update Date:	2018-05-01
Keywords:

Solution Type Problem Resolution Sure

Solution 1612672.1 : SPARC T5 System :FMA reports SUNOS-8000-FU or SUNOS-8000-J0 due to ereport.io.pciex.rc.ce-msg

Applies to:

Solaris SPARC Operating System - Version 11.1 to 11.1 [Release 11.0]
SPARC T5-2 - Version All Versions to All Versions [Release All Releases]
SPARC T5-4 - Version All Versions to All Versions [Release All Releases]
SPARC T5-8 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

SPARC T5 Systems installed with S10u11 or S11.1 with non UTF-8 Localization(LANG) may encounter the following FMA errors. ( SUNOS-8000-FU or SUNOS-8000-J0)

Example 1

--------------- ------------------------------------  -------------- ---------
TIME            EVENT-ID                              MSG-ID         SEVERITY
--------------- ------------------------------------  -------------- ---------
Jul 12 17:18:26 3408e811-f641-6290-e29f-88d360a954b8  SUNOS-8000-FU  Major    

Host        : server
Platform    : sun4v-platform Chassis_id  :
Product_sn  :

Fault class : defect.sunos.eft.undiag.fme
FRU         : None faulty

Description : The diagnosis engine encountered telemetry for which it was
             unable to perform a diagnosis.

Response    : Error reports have been logged for examination.

Impact      : Automated diagnosis and response for these events will not occur.

Action      : Use 'fmadm faulty' to provide a more detailed view of this event.
             Use 'fmdump -eV' to view the unexpected telemetry. Please refer
             to the associated reference document at
             http://sun.com/msg/SUNOS-8000-FU for the latest service
             procedures and policies regarding this diagnosis.



Example 2

--------------- ------------------------------------  -------------- ---------
TIME            EVENT-ID                              MSG-ID         SEVERITY
--------------- ------------------------------------  -------------- ---------
Aug 25 00:26:48 1f3e2835-9d6e-418f-da7a-86e405c433b7  SUNOS-8000-J0  Major   
 
Problem Status    : resolved
Diag Engine       : eft / 1.16
System
    Manufacturer  : Oracle-Corporation
    Name          : SuperCluster-T5-8
    Part_Number   : SuperCluster-T5-8
    Serial_Number : AK001XXXXX
 
System Component
    Manufacturer  : Oracle-Corporation
    Name          : SPARC-T5-8
    Part_Number   : 7068108
    Serial_Number : AK001XXXXX
    Host_ID       : 8635373e
 
----------------------------------------
Suspect 1 of 2 :
   Fault class : defect.sunos.eft.unexpected_telemetry
   Certainty   : 50%
   Resource
     Name             : "dev:////pci@340"
     Status           : repaired
----------------------------------------
Suspect 2 of 2 :
   Fault class : fault.sunos.eft.unexpected_telemetry
   Certainty   : 50%
   Resource
     Name             : "dev:////pci@340"
     Status           : repaired
 
Description : The diagnosis engine encountered telemetry from the listed
              devices for which it was unable to perform a diagnosis -
              
ereport.io.pciex.rc.ce-msg@chassis0/motherboard0/cpuboard0/chip0/hostbridge1
              class and path are incompatible.
 
Response    : Error reports have been logged for examination.
 
Impact      : Automated diagnosis and response for these events will not 
occur.
 
Action      : Use 'fmadm faulty' to provide a more detailed view of this 
event.
              Use 'fmdump -eV' to view the unexpected telemetry. Please refer
              to the associated reference document at
              http://support.oracle.com/msg/SUNOS-8000-J0 for the latest
              service procedures and policies regarding this diagnosis.

List of non utf-8 localization

Simplified Chinese LOCALE

$ grep "LANG=" /etc/default/init LANG=zh

Note: Other LANG values could be zh.GBK and zh_CN.GB18030

Traditional Chinese LOCALE

$ grep "LANG=" /etc/default/init LANG=zh_TW

Note: Other LANG values could be zh_TW.BIG5,zh_HK.BIG5HK

Japanese LOCALE

$ grep "LANG=" /etc/default/init LANG=ja

Korean LOCALE

$ grep "LANG=" /etc/default/init LANG=ko

Cause

This issue is not a hw issue and has been root caused to fma topology xml enconding and parsing issue.

A permanent fix has been released for Solaris 10 under patch 150535-01 and S11.1 SRU 14 and above 


Some of the older locallizations are non UTF-8 compliant so they break the FMA XML topology header. 
The fix was to remove localization setlocale() from "/src/lib/fm/topo/libtopo/common/topo_2xml.c", 
when FMA is creating the topology XML file. The FIX is integrated into Solaris 11.1 SRU 14 
(but not documented in the SRU notes) and Solaris 10 Patch 150535-01.

Solution

The current workaround is to set LANG=C in FMA service

STEP 1 ) Use the SMF service configuration utility [svccfg] to change the language setting on FMD by doing the following

# svccfg -s fmd
svc:/system/fmd> setenv LANG C
svc:/system/fmd> exit
#

 
STEP 2 ) Update the configuration and restart FMD by doing the following

# svcadm refresh fmd
# svcadm restart fmd

STEP 3 ) Verify the FMD configuration

# echo '::getenv -t' | mdb -p `pgrep fmd` | grep LANG
LANG=C
#

References

<BUG:17408811> - SUNOS-8000-J0 FOR EREPORT.IO.PCIEX.RC.CE-MSG ON SUPERCLUSTER T5-8
<BUG:17210614> - SPARC T5-2: SOLARIS 10 U11 FMA REPORT SUNOS-8000-FU : EREPORT.IO.PCIEX.RC.CE-MSG

Attachments

This solution has no attachment