Oracle ZFS Storage Appliance: How to Resolve Multiple Disk Failures caused by FMA SMART 'Trip'

Asset ID:	1-72-2270560.1
Update Date:	2017-11-17
Keywords:

Solution Type Problem Resolution Sure

Solution 2270560.1 : Oracle ZFS Storage Appliance: How to Resolve Multiple Disk Failures caused by FMA SMART 'Trip'

Applies to:

Oracle ZFS Storage ZS3-BA - Version All Versions to All Versions [Release All Releases]
Oracle ZFS Storage ZS3-4 - Version All Versions to All Versions [Release All Releases]
Oracle ZFS Storage ZS4-4 - Version All Versions to All Versions [Release All Releases]
Oracle ZFS Storage ZS5-2 - Version All Versions to All Versions [Release All Releases]
Oracle ZFS Storage ZS5-4 - Version All Versions to All Versions [Release All Releases]
7000 Appliance OS (Fishworks)

Symptoms

This KM is for multiple disk failures of "SMART health-monitoring firmware reported that a disk failure is imminent." that are not related to the 'Hitachi 600/900GB Drive' issue outlined in Doc ID 2073735.1.

Reported against HGST-H7280A520SUN8.0T and HGST-H101812SFSUN1.2T in ORACLE-DE2-24C and ORACLE-DE2-24P JBODS

The drives are reported as "faulted" in the hardware configuration.

FMA reports:

Fault class : fault.io.disk.predictive-failure

Description : SMART health-monitoring firmware reported that a disk failure is imminent.

BUT the disks are marked as "DEGRADED" in zpool status output (if you see "UNAVAILABLE" then this is a "real" failure) and the drives are reported as :

c0t5000CCA2604AEEA0d0 DEGRADED too many errors
status: FMA has degraded this device.
action: Run 'fmadm faulty' for more information. Clear the errors using 'fmadm repaired'.

Then this document may apply to your case.

Cause

Firmware of the drives is not compatible with the Appliance Release FMA version -> FMA will SMART 'trip' the disks if the Appliance Firmware Release is below micro release AK 8.6.12

Solution

Upgrade to Appliance Firmware Release OS 8.7.0 or later.

(Note: The Appliance software upgrade can be 'forced' - if any hardware components are in a 'non-optimal state - by using 'upgrade -f' at the CLI)

References

<NOTE:2045833.1> - Oracle ZFS Storage Appliance: Fault LED Still On - But No Faults Seen On System
<NOTE:2021771.1> - Oracle ZFS Storage Appliance: Software Updates
<NOTE:1400578.1> - Sun Storage 7000 Unified Storage System: How to collect SP/ILOM diagnostic data

Attachments

This solution has no attachment