Aura 1.x flash DOMs require firmware update in storage servers in Exadata Database Machines X2, V2, and SPARC SuperCluster T4-4 systems

Asset ID:	1-79-1504776.1
Update Date:	2016-03-21
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 1504776.1 : Aura 1.x flash DOMs require firmware update in storage servers in Exadata Database Machines X2, V2, and SPARC SuperCluster T4-4 systems

Applies to:

SPARC SuperCluster T4-4 - Version All Versions and later
Exadata Database Machine V2 - Version All Versions and later
Exadata Database Machine X2-2 Hardware - Version All Versions and later
Information in this document applies to any platform.

Purpose

Recent Aura 1.x flash DOMs in Exadata Database Machine X2 and V2 and SPARC SuperCluster T4-4 storage servers may prematurely fail if using firmware version D20Y or earlier. Aura 1.x flash DOM firmware version D20Y or earlier is supplied with Exadata Storage Server software version 11.2.3.2.0 and earlier. Firmware on recent Aura 1.x flash DOMs require firmware version D21Y. Firmware version D21Y is supplied with Exadata 11.2.3.2.1 and later, or via <Patch 14793859> for Exadata 11.2.3.2.0 and earlier.

If you plan to perform Exadata software upgrade to 11.2.3.2.0 or earlier in conjunction with install of patch 14793859 and/or patch 16042459 (to address Exadata critical issues EX9 and EX11, respectively), refer to additional guidelines in <Document 1518790.1>.

Required Actions

Action that requires immediate attention

If a recent Aura 1.x flash DOM that requires firmware version D21Y but currently has D20Y, then the firmware must be updated with urgency by upgrading to Exadata 11.2.3.2.1 or later, or by installing <Patch 14793859> on Exadata 11.2.3.2.0 and earlier. Follow the steps below to identify Aura 1.x flash DOMs that require firmware update to D21Y.

Action for next scheduled maintenance

If there are currently no Aura 1.x flash DOMs that require firmware update, then upgrade to Exadata 11.2.3.2.1 or later, or install <Patch 14793859> on Exadata 11.2.3.2.0 and earlier, during the next scheduled maintenance to prepare for the possibility of future Aura 1.x flash DOM replacement that requires firmware D21Y. If there is a flash DOM replacement before the next scheduled maintenance when running Exadata 11.2.3.2.0 and earlier, <Patch 14793859> must be installed prior to performing hardware service.

Scope

This procedure applies to storage servers that contain Aura 1.x flash cards, which are the following storage servers models only:

SUN FIRE X4270 M2 storage servers (for Exadata X2-2, X2-8, and SPARC SuperCluster systems)
SUN FIRE X4275 storage servers (for Exadata V2 systems)

The storage server model can be determined with the following CellCLI command:

CellCLI> list cell attributes makeModel
Oracle Corporation SUN FIRE X4270 M2 SERVER SAS

This procedure applies to Exadata version 11.2.3.2.0 and earlier. Application of <Patch 14793859> does not change the Exadata version.

Note that <Patch 14793859> also contains updated ILOM firmware version 3.0.16.10.d for SUN FIRE X4270 M2 storage servers and SUN FIRE X4170 M2 (X2-2) database servers.

Details

Step 1 - Identify storage servers with Aura 1.x Flash DOMs that require immediate firmware update to D21Y

Flash DOMs that require firmware update to D21Y contain date code 1219 or later. The date code is the first 4 digits of the flash disk serial number ID. The flash disk serial number ID containing the date code and the currently installed firmware version are obtained with the following CellCLI command:

CellCLI> list physicaldisk attributes name, id, physicalFirmware where diskType = 'FlashDisk'
         FLASH_1_0       1219M03F9V      D20Y
         FLASH_1_1       1025M03F9U      D20Y
         FLASH_1_2       1025M03H9F      D20Y
         FLASH_1_3       1025M03F9T      D20Y
         FLASH_2_0       1025M03G5D      D20Y
         FLASH_2_1       1025M03G6P      D20Y
         FLASH_2_2       1025M03G5G      D20Y
         FLASH_2_3       1025M03G5C      D20Y
         FLASH_4_0       1044M04NPH      D20Y
         FLASH_4_1       1041M04G29      D20Y
         FLASH_4_2       1041M04G2G      D20Y
         FLASH_4_3       1039M04EP8      D20Y
         FLASH_5_0       1025M03GKA      D20Y
         FLASH_5_1       1025M03GLJ      D20Y
         FLASH_5_2       1025M03GK9      D20Y
         FLASH_5_3       1025M03GK7      D20Y

The example above shows that flash DOM FLASH_1_0 has serial number ID 1219M03F9V. The date code is 1219. A flash DOM with date code of 1219 or later must have firmware version D21Y installed. Immediate action is required to upgrade to Exadata 11.2.3.2.1 or install <Patch 14793859>.

The CellCLI command above must be run on all storage servers.

If no storage servers contain Aura 1.x flash DOMs that require immediate firmware update, then upgrade to Exadata 11.2.3.2.1 or later, or install <Patch 14793859> on Exadata 11.2.3.2.0 and earlier during the next scheduled maintenance to prepare for the possibility of future Aura 1.x flash DOM replacement that requires firmware D21Y. If there is a flash DOM replacement before the next scheduled maintenance when running Exadata 11.2.3.2.0 and earlier, <Patch 14793859> must be installed prior to performing hardware service.

The following command may be run directly from the operating system command line:

# while read; do echo -n $REPLY; set -- $REPLY; sg_inq $(eval echo \$$#) | grep serial; done< <(lsscsi | grep MARVELL)
[8:0:0:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdn Unit serial number: 1146M0ANMC
[8:0:1:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdo Unit serial number: 1146M0ANL7
[8:0:2:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdp Unit serial number: 1146M0ANLD
[8:0:3:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdq Unit serial number: 1146M0ANMD
[9:0:0:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdr Unit serial number: 1148M0AV26
[9:0:1:0] disk ATA MARVELL SD88SA02 D20Y /dev/sds Unit serial number: 1148M0AV2A
[9:0:2:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdt Unit serial number: 1148M0AV27
[9:0:3:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdu Unit serial number: 1148M0AV2E
[10:0:0:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdv Unit serial number: 1148M0AP9D
[10:0:1:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdw Unit serial number: 1148M0AP9C
[10:0:2:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdx Unit serial number: 1148M0AP9J
[10:0:3:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdy Unit serial number: 1148M0AP6L
[11:0:0:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdz Unit serial number: 1148M0AT60
[11:0:1:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdaa Unit serial number: 1148M0AT65
[11:0:2:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdab Unit serial number: 1148M0AT5T
[11:0:3:0] disk ATA MARVELL SD88SA02 D20Y /dev/sdac Unit serial number: 1148M0AT63

Checking All Storage Servers with dcli

For systems with a large number of storage servers, the following example bash command that uses dcli may be run from a database server to query all storage servers and report only the ones that require attention:

[root@dm01db01 ~]$ while read cell name serial firmware; do
if [[ -z $firmware ]]; then
    echo "[WARNING] Unable to verify FlashDisk date code and firmware version on cell $cell - manual check required"
    continue
fi
if [[ $firmware < D21Y ]]; then
    if [[ $serial > 1219000000 ]]; then
      echo "[WARNING] Install patch to update FlashDisk firmware on cell $cell ($name, $serial, $firmware)"
    else
      echo "[INFO] Install patch to prepare for future FlashDisk replacement on cell $cell ($name, $serial, $firmware)"
    fi
fi
done < <(dcli -l root -g cell_group \
         cellcli -e "list physicaldisk \
         attributes name, id, physicalFirmware \
         where diskType = 'FlashDisk'")

The example command above requires the following:

bash shell is used on the system where the command is run
dcli command is installed
A file called cell_group exists and contains a list of all storage servers
The OS user running the command has ssh equivalence configured to the root user on all storage servers

To verify or setup ssh equivalence, refer to section 3.2 in the README for <Patch 14793859>.

If the command above produces the following output, then upgrade to Exadata 11.2.3.2.1 or <Patch 14793859> must be installed with urgency on the storage server identified:

[WARNING] Install patch to update FlashDisk firmware on cell dm01cel01: (FLASH_1_0, 1219M03F9V, D20Y)

If the command above produces the following output, then upgrade to Exadata 11.2.3.2.1 or install <Patch 14793859> during the next scheduled maintenance or prior to any Aura 1.x flash DOM replacement:

[INFO] Install patch to prepare for future FlashDisk replacement on cell dm01cel01: (FLASH_0_1, 1003M023NF, D20Y)

If the command above produces the following output, then the CellCLI command at the beginning of Step 1 must be run manually on the storage server:

[WARNING] Unable to verify FlashDisk date code and firmware version on cell dm01cel01: - manual check required

Step 2 - Upgrade to Exadata 11.2.3.2.1 (recommended) or Install <Patch 14793859>

Exadata 11.2.3.2.1 contains the fixes from patch 14793859, plus additional fixes. The recommended action is to upgrade to Exadata 11.2.3.2.1 or later.

If upgrade from the current Exadata release is not possible, then fully review the patch README, then follow the instructions in the README to install the patch.

References

<NOTE:1518790.1> - Considerations for installing Exadata Storage Server patches 14793859 and 16042459 in conjunction with Exadata upgrade and deployment

Attachments

This solution has no attachment