Sun Fire E6900 Server: How to identify a Data Center AC input power failure verses a regular domain reboot/outage

Asset ID:	1-71-1580097.1
Update Date:	2017-10-11
Keywords:

Solution Type Technical Instruction Sure

Solution 1580097.1 : Sun Fire E6900 Server: How to identify a Data Center AC input power failure verses a regular domain reboot/outage

Applies to:

Sun Fire E6900 Server - Version All Versions to All Versions [Release All Releases]
Sun Fire 6800 Server - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Goal

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in an appropriate
My Oracle Support Community - Oracle Sun Technologies Community.

How to identify a Data Center AC input power failure verses a regular domain reboot/outage when a server Rebooted Unexpectedly

Solution

1. AC input failures seen on all PSU are rare
- This is indication that we lost the main AC to all PSx

Aug 28 15:41:25 sc0 Platform.SC: [ID 354474 local0.error] AC input failure on PS1
Aug 28 15:41:27 sc0 Platform.SC: [ID 354473 local0.error] AC input failure on PS0
Aug 28 15:41:31 sc0 Platform.SC: [ID 354475 local0.error] AC input failure on PS2
Aug 28 15:41:33 sc0 Platform.SC: [ID 354476 local0.error] AC input failure on PS3
Aug 28 15:41:35 sc0 Platform.SC: [ID 354477 local0.error] AC input failure on PS4
Aug 28 15:41:37 sc0 Platform.SC: [ID 354478 local0.error] AC input failure on PS5

2. The SC uptime is 0 days which indicates a reboot/reset

- This is a good indication as normal domain outages do not reboot the SC

- The SC(s) draw power directly from AC and not the PSU.

Aug 28 15:40:57 sc1 Platform.SC: [ID 115377 local0.notice] Boot: ScApp 5.20.7, RTOS 47
Aug 28 15:40:59 sc1 Platform.SC: [ID 945168 local0.notice] SBBC Reset Reason(s): Power On Reset
Aug 28 15:40:59 sc1 Platform.SC: [ID 183033 local0.notice] Initializing the SC SRAM
Aug 28 15:41:03 sc1 Platform.SC: [ID 595276 local0.notice] Clock Source: 75MHz
Aug 28 15:41:16 sc1 Platform.SC: [ID 819635 local0.notice] Spare System Controller
Aug 28 15:41:17 sc1 Platform.SC: [ID 942133 local0.notice] Starting telnet server ...
Aug 28 15:41:17 sc1 Platform.SC: [ID 151208 local0.notice] SC Failover: enabled but not active.
Aug 28 16:22:16 sc1 Platform.SC: [ID 913363 local0.notice] SC Failover: enabled and active.
Aug 28 16:32:54 sc1 Platform.SC: [ID 381978 local0.notice] Spare, up 0 days 00:52:00, Memory 1,920,592
Aug 28 20:32:54 sc1 Platform.SC: [ID 528554 local0.notice] Spare, up 0 days 04:52:00, Memory 1,878,656

3. The SC environment logs indicate no PSU or FT failure

sc0:SC> showenvironment -ltuv

System Controller Board
Slot    Device     Sensor       Min    LoWarn Value HiWarn Max    Units     Age     Status
------- ---------- ------------ ------ ------ ------ ------ ------ --------- ------- ------
SSC0    SBBC 0     Temp. 0           0      0 47        100    105 Degrees C   6 sec OK
SSC0    CBH 0      Temp. 0           0      0 47        100    105 Degrees C   6 sec OK
SSC0    SSC 0      Temp. 0           0      0 34         85     90 Degrees C   6 sec OK
SSC0    SSC 0      Temp. 1           0      0 30         85     90 Degrees C   6 sec OK
SSC0    SSC 0      Temp. 2           0      0 33         85     90 Degrees C   6 sec OK
SSC0    SSC 0      1.5 VDC 0      1.35   1.42 1.52     1.57   1.65 Volts DC    6 sec OK
SSC0    SSC 0      3.3 VDC 0      2.97   3.13 3.33     3.46   3.63 Volts DC    6 sec OK
SSC0    SSC 0      5 VDC 0        4.50   4.75 4.98     5.25   5.50 Volts DC    6 sec OK

A184 Power Supply
Slot    Device     Sensor       Min    LoWarn Value HiWarn Max    Units     Age     Status
------- ---------- ------------ ------ ------ ------ ------ ------ --------- ------- ------
PS0     48 VDC 0   Current 0     -1.00 -1.00 14.39   33.00 45.00 Amps        5 sec OK
PS0     48 VDC 0   48 VDC 0      40.00 40.00 56.07   60.00 60.00 Volts DC    5 sec OK
PS0     48 VDC 0   Temp. 0          -1     -1 26         65     78 Degrees C   5 sec OK
PS0     48 VDC 1   Current 0     -1.00 -1.00 0.23     1.45   1.55 Amps        5 sec OK
PS0     48 VDC 1   48 VDC 0      40.00 40.00 55.02   60.00 60.00 Volts DC    5 sec OK
PS1     48 VDC 0   Current 0     -1.00 -1.00 14.83   33.00 45.00 Amps        2 sec OK
PS1     48 VDC 0   48 VDC 0      40.00 40.00 56.59   60.00 60.00 Volts DC    2 sec OK
PS1     48 VDC 0   Temp. 0          -1     -1 26         65     78 Degrees C   2 sec OK
PS1     48 VDC 1   Current 0     -1.00 -1.00 0.12     1.45   1.55 Amps        2 sec OK
PS1     48 VDC 1   48 VDC 0      40.00 40.00 55.28   60.00 60.00 Volts DC    2 sec OK
PS2     48 VDC 0   Current 0     -1.00 -1.00 14.16   33.00 45.00 Amps        1 sec OK
PS2     48 VDC 0   48 VDC 0      40.00 40.00 56.33   60.00 60.00 Volts DC    1 sec OK
PS2     48 VDC 0   Temp. 0          -1     -1 26         65     78 Degrees C   1 sec OK
PS2     48 VDC 1   Current 0     -1.00 -1.00 0.08     1.45   1.55 Amps        1 sec OK
PS2     48 VDC 1   48 VDC 0      40.00 40.00 54.76   60.00 60.00 Volts DC    1 sec OK
PS3     48 VDC 0   Current 0     -1.00 -1.00 14.26   33.00 45.00 Amps        4 sec OK
PS3     48 VDC 0   48 VDC 0      40.00 40.00 56.07   60.00 60.00 Volts DC    4 sec OK
PS3     48 VDC 0   Temp. 0          -1     -1 26         65     78 Degrees C   4 sec OK
PS3     48 VDC 1   Current 0     -1.00 -1.00 0.21     1.45   1.55 Amps        4 sec OK
PS3     48 VDC 1   48 VDC 0      40.00 40.00 54.76   60.00 60.00 Volts DC    4 sec OK
PS4     48 VDC 0   Current 0     -1.00 -1.00 14.34   33.00 45.00 Amps        4 sec OK
PS4     48 VDC 0   48 VDC 0      40.00 40.00 56.07   60.00 60.00 Volts DC    4 sec OK
PS4     48 VDC 0   Temp. 0          -1     -1 26         65     78 Degrees C   4 sec OK
PS4     48 VDC 1   Current 0     -1.00 -1.00 0.12     1.45   1.55 Amps        3 sec OK
PS4     48 VDC 1   48 VDC 0      40.00 40.00 55.02   60.00 60.00 Volts DC    3 sec OK
PS5     48 VDC 0   Current 0     -1.00 -1.00 13.72   33.00 45.00 Amps        4 sec OK
PS5     48 VDC 0   48 VDC 0      40.00 40.00 56.07   60.00 60.00 Volts DC    4 sec OK
PS5     48 VDC 0   Temp. 0          -1     -1 26         65     78 Degrees C   4 sec OK
PS5     48 VDC 1   Current 0     -1.00 -1.00 0.17     1.45   1.55 Amps        3 sec OK
PS5     48 VDC 1   48 VDC 0      40.00 40.00 55.02   60.00 60.00 Volts DC    3 sec OK

Fan Tray (F6800, I/O)
Slot    Device     Sensor       Min    LoWarn Value HiWarn Max    Units     Age     Status
------- ---------- ------------ ------ ------ ------ ------ ------ --------- ------- ------
FT0     Fan 0      Cooling 0                  Low                              5 sec OK

Fan Tray (F6800, CPU)
Slot    Device     Sensor       Min    LoWarn Value HiWarn Max    Units     Age     Status
------- ---------- ------------ ------ ------ ------ ------ ------ --------- ------- ------
FT1     Fan 0      Cooling 0                  Low                              4 sec OK

Fan Tray (F6800, I/O)
Slot    Device     Sensor       Min    LoWarn Value HiWarn Max    Units     Age     Status
------- ---------- ------------ ------ ------ ------ ------ ------ --------- ------- ------
FT2     Fan 0      Cooling 0                  Low                              3 sec OK

Fan Tray (F6800, CPU)
Slot    Device     Sensor       Min    LoWarn Value HiWarn Max    Units     Age     Status
------- ---------- ------------ ------ ------ ------ ------ ------ --------- ------- ------
FT3     Fan 0      Cooling 0                  Low                              3 sec OK

4. All of the above are indicators that the platform had an external AC input failure

- The E6900 class of servers use Dual Power GRIDS and have redundant RTU/RTS

- if all PSU are failing, this is a good indication of both GRIDS failing from a data center outage

Data center operations do have power log either from the UPS or some power monitor devices
Verify against the timing of AC input failure should have a perfect match.

References

<NOTE:1007049.1> - Sun Fire [TM] SF3800/SF4800/SF4810/SF6800 - E4900/E6900 - V1280/E2900 - Netra 1280/1290 : Troubleshooting Power Supply Warnings or Failures
<NOTE:1011893.1> - Sun Fire [TM] 3800, 4800, 4810, E4900, 6800, E6900 AC/DC Power Distribution
<NOTE:1009300.1> - Sun Fire[TM] 4800/4810/4900/6800/6900 servers: Fault LED on power supply

Attachments

This solution has no attachment