Exadata X5-8 Hardware
 
Exadata X5-2 Hardware
 
Exadata X4-8 Hardware
 
Oracle SuperCluster M6-32 Hardware
 
Oracle SuperCluster T5-8 Hardware
 

PLA-Support>Sun Systems>x86>Engineered Systems HW>SN-x64: EXADATA
 

SAS HBA does not maintain logs over a reboot on Exadata X5-2L High Capacity Storage Servers on SW Image versions below 12.1.2.3.0

In this Document

Symptoms

Cause

Solution

References

Applies to:

Symptoms

SAS HBA does not maintain logs over a reboot on Exadata X5-2L High Capacity Storage Servers on SW Image versions below 12.1.2.3.0.

Note: Exadata X5-2L Extreme Flash Storage Servers do not have a SAS HBA and are not affected.
Note: Exadata systems older than X5-2 use a different SAS HBA and are not affected.

Cause

Firmware bug and default log setting changes.

Solution

1. Check the image version of the system and firmware version of the SAS HBA: 

If the system is running SW Image 12.1.2.3.0 or later, then the problem does not apply. These images have both the firmware fix and the persistent log setting is already enabled by default, so no further action is required.

If the system is running SW Image 12.1.2.2.x, then these images have the firmware fix 24.3.0-0081 per the example above, but the HBA configuration settings need to be updated. Proceed to step 2.

If the system is running SW Image release earlier than 12.1.2.2.0 with firmware 24.3.0-0073 per the example above, then it needs to be updated to SW Image 12.1.2.2.0 or later, containing SAS HBA firmware package 24.3.0-0081 due to a firmware bug with terminal logging.  For how to update image, refer to MOS Note 888828.1.  The first preference for resolving this issue is to update image. If the server is not able to be updated with a later image at this time, then the SAS HBA firmware only may be temporarily updated to address this issue on systems that have had a failure as follows, using the firmware package "MR_6.3.8_24.3.0-0081.rom" attached to this Note.

a) Download the firmware package "MR_6.3.8_24.3.0-0081.rom" attached to this Note, and copy it to the /tmp directory on the Storage Cell.

b) Prepare the High Capacity Storage Cells for maintenance as follows:

i. ASM drops a disk shortly after it/they are taken offline. The default DISK_REPAIR_TIME
attribute value of 3.6hrs should be adequate for replacing components, but may have been
changed by the Customer. To check this parameter, have the Customer log into ASM and
perform the following query:

As long as the value is large enough to comfortably replace the hardware in a
storage cell, there is no need to change it.

ii. Check if ASM will be OK if the grid disks go OFFLINE.

If one or more disks return asmdeactivationoutcome='No', you should wait for some time
and repeat the query until all disks return asmdeactivationoutcome='Yes'.

iii. Run cellcli command to Inactivate all grid disks on the cell you wish to power down/reboot.
(this could take up to 10 minutes or longer)

iv. Execute the command below and the output should show asmmodestatus='UNUSED' or
'OFFLINE' and ‘asmdeactivationoutcome=Yes’ for all griddisks once the disks are offline and
inactive in ASM.

c) Disable Exadata Storage Server services with the following command as 'root' user:

d) Upgrade the HBA firmware with the following command as 'root' user:

   Upon completion of the firmware upgrade, the system will automatically reboot. This takes ~10 minutes to complete the entire process after rebooting the cell, excluding disk re-synchronization time.

e) Verify the SAS HBA firmware is updated:

 The firmware package with the logging bug fix is 24.3.0-0081.

f) Verify the disks and bring them online as follows:

i. Verify the 12 disks are visible. The following command should show 12 disks:

ii. Activate the grid disks.

iii. Verify all grid disks show 'active':

iv. Verify all grid disks have been successfully put online using the following command. Wait until asmmodestatus is ONLINE for all grid disks. The following is an example of the output early in the activation process.

Notice in the above example that RECO_CD_00_dmorlx8cel01 is still in the 'SYNCING' process. Oracle ASM synchronization is only complete when ALL grid disks show ‘asmmodestatus=ONLINE’. This process can take some time depending on how busy the machine is, and has been while this individual server was down for repair.

g) Repeat the above steps to update the firmware on each storage cell, as needed.

2. To verify the current battery status for the fwtermlog setting, on each storage cell, as 'root' login, execute:  

We should see that the battery mode is off for the fwtermlog.


3. Turn on use of the battery for maintaining the fwtermlog across cell reboots and power cycles:

Running the above command on the cells will not have any impact.

This change is persistent across cell reboots or power cycles and is only unset by command.

References

This solution has no attachment