How To Resolve Unrecoverable USB Hardware Error Reported On Internal USB For ZFS Storage Appliance

Asset ID:	1-71-2018621.1
Update Date:	2018-05-11
Keywords:

Solution Type Technical Instruction Sure

Solution 2018621.1 : How To Resolve Unrecoverable USB Hardware Error Reported On Internal USB For ZFS Storage Appliance

Applies to:

Exalogic Elastic Cloud X4-2 Hardware - Version X4 to X4 [Release X4]
Exalogic Elastic Cloud X5-2 Half Rack - Version X5 to X5 [Release X5]
Exalogic Elastic Cloud X4-2 Eighth Rack - Version X4 to X4 [Release X4]
Oracle ZFS Storage ZS3-2 - Version All Versions to All Versions [Release All Releases]
Exalogic Elastic Cloud X3-2 Full Rack - Version X3 to X3 [Release X3]
7000 Appliance OS (Fishworks)

Goal

A number of ZFS-ES systems have encountered the following message on installation:

Description : A hardware fault within the device or its interface was detected
in the USB device. The driver has failed to initialize the device
and the device is in an invalid state.

Response : Device has been disabled and is not functional.

Impact : Loss of services provided by the device instances associated with this fault.

Solution

This message is generated because the ZFS ES NAS does not use the USB device - as fitted to the base server.

The action is to ignore the message, as it should only appear once.

If ignoring the message is not appropriate then log a Support ticket for TSC to correct.

See also:

Doc ID 2042348.1 - Oracle ZFS Storage Appliance: How to recover from a USB fault on a ZS3-X, if 'Mark As Repaired' does not work

NOTE: Verification required but 3.4.2.26 the osa option was greyed out in one instance - lab machine shows the option as expected though

Here is what to do to disable the internal USB ports

advance > select USB Ports

| Port 60/64 Emulation [Enabled] |EHCI ownership change |
| |should be claimed by |
| All USB Devices [Enabled] |EHCI driver. |
| Rear Port 0 [Enabled] | |
| Rear Port 1 [Enabled] | |
| Front Port 0 [Enabled] |-------------------------|
| Front Port 1 [Enabled] |><: Select Screen |
| Internal Port 0 [Disabled] |^v: Select Item |
| Internal Port 1 [Disabled] |Enter: Select |

These internal methods have been tried

1) Disable the option from BIOS ( normally there's no USB stick there )

    Get into BIOS boot option>
    OSA Configuration > ## navigate to osa screen
    OSA internal support > Toggle the option to disable

2) Clear FMA as per bug using the fmadm rotate commands and then disabling FMA as follows

        # fmadm rotate errlog
        fmadm: errlog has been rotated out and can now be archived
        # fmadm rotate fltlog
        fmadm: fltlog has been rotated out and can now be archived
        # fmadm rotate infolog
        fmadm: infolog has been rotated out and can now be archived
        # fmadm rotate infolog_hival
        fmadm: infolog_hival has been rotated out and can now be archived
        # svcadm disable -t fmd
        # cd /var/fm/fmd/rsrc/
        # rm *
        # svcadm enable fmd
        # fmadm faulty

NOTE : the rm * will remove ALL faults even if they are unresolved , alternativly you can use the UUID from fmadm faulty to determine live issues from old unrepaired ones

NOTE2 : It has been reported that you may still have to run "fmadm acquit <uuid>" to get rid of the unwanted fm message

3) "cfgadm method" - a temporary method of disabling the device that will NOT survive a reboot

Example:

cfgadm -l |grep -v empty
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             scsi-bus     connected    configured   unknown
usb0/1                         usb-hub      connected    configured   ok
usb0/1.2                       usb-storage connected    configured   ok
usb1/1                         usb-hub      connected    configured   ok
usb1/1.7                       usb-hub      connected    configured   ok
usb1/1.7.3                     usb-communications connected    configured   ok
usb1/1.8                       usb-device   connected    configured   ok

Ap_Id                          Receptacle   Occupant     Condition Information
When         Type         Busy     Phys_Id
unavailable scsi-bus     n        /devices/pci@0,0/pci8086,e0a@3,2/pci1000,9263@0:scsi
usb0/1                         connected    configured   ok         Mfg: <undef> Product: <undef> NConfigs: 1 Config: 0 <no cfg str descr>
unavailable usb-hub      n        /devices/pci@0,0/pci108e,4856@1a:1
usb0/1.2                       connected    configured   ok         Mfg: ORACLE Product: SSM NConfigs: 1 Config: 0 <no cfg str descr>
unavailable usb-storage n        /devices/pci@0,0/pci108e,4856@1a/hub@1:1.2 <<<<< This guy is the 3.8 g usb internal disk that we want to disable>>
usb1/1                         connected    configured   ok         Mfg: <undef> Product: <undef> NConfigs: 1 Config: 0 <no cfg str descr>
unavailable usb-hub      n        /devices/pci@0,0/pci108e,4856@1d:1
usb1/1.7                       connected    configured   ok         Mfg: American Megatrends Inc. Product: Generic Hub NConfigs: 1 Config: 0 <no cfg str descr>
unavailable usb-hub      n        /devices/pci@0,0/pci108e,4856@1d/hub@1:1.7
usb1/1.7.3                     connected    configured   ok         Mfg: SunMicro Product: Virtual Eth Device NConfigs: 2 Config: 1 : CDC Ethernet Configuration
unavailable usb-communications n        /devices/pci@0,0/pci108e,4856@1d/hub@1/hub@7:1.7.3
usb1/1.8                       connected    configured   ok         Mfg: American Megatrends Inc. Product: Virtual Keyboard and Mouse NConfigs: 1 Config: 0 <no cfg str descr>
unavailable usb-device   n        /devices/pci@0,0/pci108e,4856@1d/hub@1:1.8

# hotplug list -dlv | grep /pci108e,4856@1a
/pci@0,0/pci108e,4856@1a ehci#0
/pci@0,0/pci108e,4856@1a/hub@1 hubd#1
/pci@0,0/pci108e,4856@1a/hub@1/storage@2 scsa2usb#0
/pci@0,0/pci108e,4856@1a/hub@1/storage@2/disk@0,0 sd#1

# cfgadm -c unconfigure usb0/1.2
Unconfigure the device: /devices/pci@0,0/pci108e,4856@1a/hub@1:1.2
This operation will suspend activity on the USB device
Continue (yes/no)? yes

(Re-confirmed the same with hotplug)

# hotplug list -dlv | grep /pci108e,4856@1a
/pci@0,0/pci108e,4856@1a ehci#0
/pci@0,0/pci108e,4856@1a/hub@1 hubd#1
/pci@0,0/pci108e,4856@1a/hub@1/storage@2 scsa2usb#0

Disabling from BIOS << Permanent Fix>>>

# hotplug list -dlv | grep /pci108e,4856@1a
/pci@0,0/pci108e,4856@1a ehci#0
/pci@0,0/pci108e,4856@1a/hub@1 hubd#1

NOTE: It has been seen that all these methods failed and the fault had to be cleared on the SP/ILOM with SP faultmgmt

Check for relevancy - 11-May-2018

References

<BUG:20957047> - UNRECOVERABLE USB HARDWARE ERROR REPORTED ON INTERNAL USB

Attachments

This solution has no attachment