Exadata: USB Errors: [ERROR] USB is not fixable

Asset ID:	1-72-2301269.1
Update Date:	2017-08-28
Keywords:

Solution Type Problem Resolution Sure

Solution 2301269.1 : Exadata: USB Errors: [ERROR] USB is not fixable

Applies to:

Exadata X5-2 Quarter Rack - Version All Versions and later
Information in this document applies to any platform.

Symptoms

Alerthistory shows the below error :

Internal USB status is degraded. Affected USB Slots : 0

Imageinfo shows below :

Kernel version: 2.6.39-400.277.1.el6uek.x86_64 #1 SMP Wed Feb 24 16:13:42 PST 2016 x86_64
Cell version: OSS_12.1.2.3.1_LINUX.X64_160411
Cell rpm version: cell-12.1.2.3.1_LINUX.X64_160411-1.x86_64

Active image version: 12.1.2.3.1.160411
Active image kernel version: 2.6.39-400.277.1.el6uek
Active image created: 2016-04-11 14:29:43 -0700
Active image activated: 2016-07-16 15:32:54 +0200
Active image type: production
Active image status: success
Active image label: OSS_12.1.2.3.1_LINUX.X64_160411
Active node type: STORAGE
Active system partition on device: /dev/md6
Active software partition on device: /dev/md8

Cell boot usb partition: /dev/sda1
Cell boot usb version: 12.1.2.3.1.160411

Cell details shows Model as X5-2

makeModel: Oracle Corporation ORACLE SERVER X5-2L_EXTREME_FLASH

ms-odl.trc file shows below :

[2017-07-14T09:58:17.701+02:00] [ossmgmt] [WARNING] [] [oracle.ossmgmt.common.hwadapter.osadp.OSAdapterImpl] [tid: 42] sendUSBAlertAndASR: usb error code - 589825
[2017-07-14T09:58:17.701+02:00] [ossmgmt] [NOTIFICATION:16] [CELL-05003] [oracle.ossmgmt.ms.core.MSCommandHandler] [tid: 42] MS detected error: CELL-02827: Configuration check for hardware and firmware encountered the following issues:[[

Cell configuration check for devices encountered the following issues:
<exadata:br/>USB Errors: <exadata:tab/>[ERROR] USB is not fixable.<exadata:tab/>[ERROR] Multiple bootable USB devices found.<exadata:tab/> -- while executing command: alter cell validate configuration

Cause

For a brief time, the X5-2 supported two USB drives. In later versions of firmware, this no longer became necessary.

If this error is seen, make the second USB non-bootable. Re-run "alter cell validate configuration" and the issue will clear.

Solution

Looking at imageinfo we see the node is using /dev/sda1 as the boot loader.

Cell boot usb partition: /dev/sda1

Check whether /dev/sdb1 does contain a boot partition

# e2label /dev/sdab1

If the above command returns the below error :

e2label: Bad magic number in super-block while trying to open /dev/sda1
Couldn't find valid filesystem superblock.

Then first attempt to make /dev/sdb1 bootable.

# cd /opt/oracle.SupportTools

# cellcli -e alter cell shutdown services MS (This doesn't impact any services)

# ./make_cellboot_usb -verbose -force

If this fails then the USB is broken and needs replacing.

if it passes then check it has a boot signature.

# e2label /dev/sdb1
CELLBOOT
#

Now run # cellcli -e 'alter cell validate configuration'

If this reports that the node now has :

CELL-02827: Configuration check for hardware and firmware encountered the
following issues:
Cell configuration check for devices encountered the following issues:

USB Errors: [ERROR] USB is not fixable. [ERROR] Multiple bootable USB devices found.

Try restarting MS and then run the validate again.

# cellcli -e alter cell startup services MS

# cellcli -e 'alter cell validate configuration'

Last resort is to reboot the cell.

In case if you need to reboot the cellnode,please follow the below mentioned note

Steps to shut down or reboot an Exadata storage cell without affecting ASM (Doc ID 1188080.1)

It should then run the validate cleanly

# cellcli -e 'alter cell validate configuration'
Cell 'cell_name' successfully altered

References

<BUG:25491107> - INTERNAL USB STATUS IS DEGRADED. AFFECTED USB SLOTS : 0,1
<NOTE:1188080.1> - Steps to shut down or reboot an Exadata storage cell without affecting ASM
<NOTE:761868.1> - Oracle Exadata Diagnostic Information required for Disk Failures and some other Hardware issues

Attachments

This solution has no attachment