Exadata Battery Backup Unit (BBU) alert: The HDD disk controller battery charge too low

Asset ID:	1-72-2166573.1
Update Date:	2018-03-26
Keywords:

Solution Type Problem Resolution Sure

Solution 2166573.1 : Exadata Battery Backup Unit (BBU) alert: The HDD disk controller battery charge too low

Applies to:

Exadata X3-2 Half Rack - Version All Versions and later
Information in this document applies to any platform.

Symptoms

Getting alert like below one :

name: 1_1
alertDescription: "THe HDD disk controller battery charge too low"
alertMessage: "The HDD disk controller battery has current capacity under threshold value of 800 mAh. All disk drives have been placed in WriteThrough caching mode. Disk write performance may be reduced. The flash drives are not affected. Battery Serial Number : 6186 Battery Type : ibbu08 Battery Temperature : 35 C Full Charge Capacity : 1220 mAh Relative Charge : 64% Ambient Temperature : 27 C"
alertSequenceID: 1
alertShortName: Hardware
alertType: Stateful
beginTime: 2016-07-12T01:11:07-04:00
examinedBy:
metricObjectName: Disk_Controller_Battery
notificationState: 1
sequenceBeginTime: 2016-07-12T01:11:07-04:00
severity: critical
alertAction: "Replace the HDD disk controller battery or schedule a learn cycle

In CellCLI alerthistory it will be displayed as below:

1_1 2016-07-12T01:11:07-04:00 critical "The HDD disk controller battery has current capacity under threshold value of 800 mAh. All disk drives have been placed in WriteThrough caching mode. Disk write performance may be reduced. The flash drives are not affected. Battery Serial Number : 6186 Battery Type : ibbu08 Battery Temperature : 35 C Full Charge Capacity : 1220 mAh Relative Charge : 64% Ambient Temperature : 27 C"

Cause

1) This alert is being generated when the Battery Back Up unit (BBU) "Remaining Capacity" is under 800 mAh.

Example:

#/opt/MegaRAID/MegaCli/MegaCli64 -adpbbucmd -aALL

Relative State of Charge: 64 %
Absolute State of charge: 51 %
Remaining Capacity: 776 mAh <--------------------------------------------- it is under 800 mAh
Full Charge Capacity: 1220 mAh <------------------------------------------ Full Charge capacity is OK
Run time to empty: Battery is not being charged.
Average time to empty: 1 Hour, 33 Min.
Estimated Time to full recharge: Battery is not being charged.
Cycle Count: 8

It does not mean that the BBU is faulty but that its current charge ("Remaining Capacity") is under the 800 mAh threshold.

2 ) How to recognize a BBU that needs replacement ?

If the BBU is faulty its "Full Charge Capacity" should be under 800 mAh (which is not the case in above example) and/or the "Max error" should be greater than 10%.

You can check the value from mentioned parameters (Full Charge Capacity" and "Max error") by running below command:

#/opt/MegaRAID/MegaCli/MegaCli64 -AdpBbuCmd -aALL | grep "Full Charge" -A5 | sort | grep Full -A1
Full Charge Capacity: 1220 mAh
Max Error: 2 %

Based on the results from above commands, you can determine if the BBU needs to be replaced or not :
Full Charge Capacity: a good battery should show greater than 800 mAh
Max Error: "Max Error" should be <10%

Solution

Once we have proved the BBU is not faulty, we need to schedule a learn cycle as mentioned in the "alertAction" in order to charge the BBU

alertAction: "Replace the HDD disk controller battery or schedule a learn cycle.

How to initiate a learn cycle ? There are 2 methods :

1) To run BBU relearn cycle manually, megacli command can be used.

# MegaCli64 -adpbbucmd -bbulearn -a0 -nolog

Note that while the relearn cycle is in progress the BBU will be fully discharged and charged,
so until the learn cycle is finish the disk cache policy will change to WriteThrough and the disk write performance will be reduced.

For more info about disk cache policy please see below doc:
Exadata Hardware Alert: All Logical Drives Are In Writethrough Caching Mode (Doc ID 1283341.1)

2) Rebooting the cell node will also launch the learn cycle.

For more info, please read below doc:
Reference information on LSI HBA battery backup unit (BBU) used on SAS2 RAID HBA (Doc ID 1542070.1)

Note also that since RAID controller firmware version 12.12.0-178, the battery does not need to charge to 100% of its "Full Charge Capacity",it will only charge to mAh needed for 2 days of data retention.

As battery ages, this percentage will get larger. This allows the unused battery to age slower. For more info please see below doc:

BBU BATTERIES DON'T FULLY CHARGE UP TO 100% OF AVAILABLE CAPACITY (Doc ID 1669359.1)

References

<NOTE:1542070.1> - Reference information on LSI HBA battery backup unit (BBU) used on SAS2 RAID HBA
<NOTE:1448073.1> - Engineered Systems Upgrade/Supportability Reference Doc (INTERNAL ONLY)
<NOTE:1669359.1> - BBU BATTERIES DON'T FULLY CHARGE UP TO 100% OF AVAILABLE CAPACITY

Attachments

This solution has no attachment