I2C bus noise causing unexpected outages on T4-4 due to PSU vendor mismatch

Asset ID:	1-72-2067274.1
Update Date:	2017-05-01
Keywords:

Solution Type Problem Resolution Sure

Solution 2067274.1 : I2C bus noise causing unexpected outages on T4-4 due to PSU vendor mismatch

Applies to:

SPARC T4-4 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

In some rare cases, a system sporting at least one power supply different from the others (Either Astec or Power One) may cause sufficient bus noise on the I2C bus to cause erroneous behaviour.

This behaviour includes, but not limited to:

Outages

Power offs

Sensor errors

Etc.

An example of erroneous power off:

11718 Tue Aug 4 04:42:56 2015 Power On major Power to /SYS has been turned on by: Shell session, Username:root
11717 Tue Aug 4 04:31:36 2015 System Log minor Host: Powered Off
11716 Tue Aug 4 04:31:36 2015 System Log minor System power off has been requested via power button. <--------------
11715 Tue Aug 4 04:31:32 2015 System Log minor Host: Host shutting down
11714 Tue Aug 4 04:31:32 2015 System Log minor System shutdown has been requested. <---------

Aug 4 10:14:31 unix: WARNING: Power-off requested, system will now shutdown. <------------
Shutdown started. Tuesday, August 4, 2015 10:14:32 AM IST
Changing to init state 5 - please wait
Broadcast Message from root (msglog) Tue Aug 4 10:14:32...
THE SYSTEM IS BEING SHUT DOWN NOW ! ! !
Log off now or risk your files being damaged
2015-08-04 04:54:49 0:0:0> NOTICE: Loaded ASR status DB data. Ver. 4.
2015-08-04 04:54:52 0:0:0> NOTICE: Initializing TPM with:
tpm_enable = false
tpm_activate = false
tpm_forceclear = false
2015-08-04 04:54:52 0:0:0> NOTICE: TPM found: Ver 1.2, Rev 1.2, SpecLevel 2, errataRev 0, VendorId 'IFX'
2015-08-04 04:54:53 0:0:0> NOTICE: TPM initialized successfully. Current state is: disabled
2015-08-04 04:54:53 2:0:0> NOTICE: Serial#: 000000000000002a.0158c4c07e0b357b
2015-08-04 04:54:53 3:0:0> NOTICE: Serial#: 000000000000002a.015a50c07e0b31aa
2015-08-04 04:54:53 0:0:0> NOTICE: Serial#: 000000000000046a.235a50c07d842b73
2015-08-04 04:54:53 1:0:0> NOTICE: Serial#: 00000000000004ea.275b58c07d842bac
2015-08-04 04:54:54 2:0:0> NOTICE: Version: 003e003013030607
2015-08-04 04:54:54 3:0:0> NOTICE: Version: 003e003013030607
2015-08-04 04:54:54 0:0:0> NOTICE: Version: 003e003012030607
2015-08-04 04:54:54 1:0:0> NOTICE: Version: 003e003012030607
2015-08-04 04:54:54 2:0:0> NOTICE: T4 Revision: 1.3

This is done without any manual intervention or any commands' input.

If additional symptoms manifest, please also look at Understanding RIO / PSU messages on a T4-4 and SPARC Super Cluster T4-4 system and checking for i2c issues (Doc ID 1986303.1)

Cause

I2C bus noise could be caused by mismatched PSU vendors.

For example:

PS0                                7048276-01                        476856F+1238CC00MD    Astec Intern                        OK  
PS1                                7048276-01                        465776G+1247A90AEH    Power-One, I                        OK  <---------------
PS2                                7048276-01                        476856F+1237CC005Y    Astec Intern                        OK  
PS3                                7048276-01                        476856F+1238CC015E    Astec Intern                        OK

Solution

First step to reduce bus noise is updating firmware. If the firmware is at a good level, then the next step would be to replace the mismatching power supply.

Should replacing the power supply not resolve the issue, further investigation would be needed.

Vendor mixing should be supported with 7060587, but if there's a mix of older part numbers (like 7048276), it should be considered as the cause of the issue.

References

<NOTE:1986303.1> - Understanding RIO / PSU messages on a T4-4 and SPARC Super Cluster T4-4 system and checking for i2c issues

Attachments

This solution has no attachment