Grid Control falsely indicates Exadata Power Supply sensor and Voltage sensor problems.

Asset ID:	1-72-1332949.1
Update Date:	2012-10-24
Keywords:

Solution Type Problem Resolution Sure

Solution 1332949.1 : Grid Control falsely indicates Exadata Power Supply sensor and Voltage sensor problems.

Applies to:

Enterprise Manager for Miscellaneous Plug-ins - Version 11.1.0.1 and later
Exadata Database Machine X2-2 Hardware - Version All Versions and later
Linux x86-64

Symptoms

After recently implementing Enterprise Manager Grid Control monitoring on an Exadata rack, you begin getting the following alerts from both the Compute Nodes and Storage Cells:

CRITICAL - Power Supply sensor #0x74 indicates that it is at state: Asserted;Power Supply sensor #0x75 indicates that it is at state: Asserted

CRITICAL - Voltage sensor #0x6d indicates that it is at state: Lower Non-recoverable

Changes

Recently implemented Enterprise Manager Grid Control monitoring.

Cause

If your current ILOM version on an affected system is the newest available, then:
You are encountering <Bug:12544896> "FALSE ALERT - CRITICAL - VOLTAGE SENSOR", which has been closed as a user error.

If your current ILOM version on an affected system is not the newest available then:
You are encountering the issue described in MOS <Document 1310539.1> "Exadata ILOM memory leak in older ILOM firmware".

To determine what the newest, available ILOM firmware version is, please refer to MOS <Document 1310539.1> "Exadata ILOM memory leak in older ILOM firmware".

Solution

Please ignore the false indications.

Workaround for now:

1.) Review the ILOM alert log and ensure that all alerts are closed out.

To ensure that all alerts are closed out, the easiest action is to address the alert itself by fixing or correcting the issue(s) listed in the ILOM event log. Alternatively, you could also manually de-assert the alert by using the ipmitool as directed in <NOTE 1398378.1>, "ILOM Targets Raise Critical Power Supply Sensor Alerts In EM That Never Clear"

2.) Then wait for the re-evaluation of the ILOM alert metric from EM (5 or 15 minutes... depending on the duration set by the user). The next time Grid Control agent runs the sensor metric it will clear the alert.

3.) You may also want to consider restarting the ILOM service processor as discussed in MOS <Document 1310539.1> "Exadata ILOM memory leak in older ILOM firmware".

References

<NOTE:1327022.1> - Update Exadata ILOM firmware manually from Storage Cell software patchset
<NOTE:1398378.1> - ILOM Targets Raise Critical Power Supply Sensor Alerts In EM That Never Clear
<BUG:12544896> - FALSE ALERT - CRITICAL - VOLTAGE SENSOR
<NOTE:1310539.1> - Exadata ILOM memory leak in older ILOM firmware

Attachments

This solution has no attachment