How to Replace a boot drive in an Oracle Database Appliance X5-2/X6-2HA/X6-2S/X6-2M Server

Asset ID:	1-71-1991446.1
Update Date:	2018-05-24
Keywords:

Solution Type Technical Instruction Sure

Solution 1991446.1 : How to Replace a boot drive in an Oracle Database Appliance X5-2/X6-2HA/X6-2S/X6-2M Server

Applies to:

Oracle Database Appliance X5-2 - Version All Versions to All Versions [Release All Releases]
Oracle Database Appliance X6-2 HA Hardware - Version All Versions to All Versions [Release All Releases]
x86_64
The OS drives are configured in a RAID1 hardware raid.
There are no md devices to sync after the disk has been replaced.

Goal

Describe the procedure needed to replace a boot drive in a server node of an Oracle Database Appliance X5-2/X6-2S/X6-2M Server.

Solution

DISPATCH INSTRUCTIONS

- WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED?

Training and experience with Oracle Database Appliance systems

TIME ESTIMATE: 30 minutes

TASK COMPLEXITY: 3

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

PROBLEM OVERVIEW: There are two OS boot drives in each server node of the Oracle Databaes Appliance X5-2. This document describes how to replace one of these drives.

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY?:

The OS boot drives are hot-swappable.

WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE?:

1. Determine which disk drive needs to be replaced.

# /opt/MegaRAID/storcli/storcli64 /c0 show
Generating detailed summary of the adapter, it may take a while to complete.

Controller = 0
Status = Success
Description = None

Product Name = LSI MegaRAID 9361-8i
Serial Number = SV50453013
SAS Address = 500605b00a08c7f0
PCI Address = 00:23:00:00
System Time = 03/18/2015 19:33:43
Mfg. Date = 01/24/15
Controller Time = 03/19/2015 01:33:44
FW Package Build = 24.3.0-0063
BIOS Version = 6.17.04.2_4.16.08.00_0x06060A05
FW Version = 4.230.40-3739
Driver Name = megaraid_sas
Driver Version = 06.803.02.00-rc1
Vendor Id = 0x1000
Device Id = 0x5D
SubVendor Id = 0x1000
SubDevice Id = 0x9361
Host Interface = PCIE
Device Interface = SAS-12G
Bus Number = 35
Device Number = 0
Function Number = 0
Drive Groups = 1

TOPOLOGY :
========

--------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type State BT       Size PDC PI SED DS3 FSpace
--------------------------------------------------------------------------
0 -   -   -        -   RAID1 Dgrd N 557.861 GB dsbl N N   none N
0 0   -   -        -   RAID1 Dgrd N 557.861 GB dsbl N N   none N
0 0   0   252:0    8   DRIVE Dgrd N 557.861 GB dsbl N N   none -    <<<<<<<<<<<<<<<<<<<<<<<< In this example, disk 0 needs to be replaced
0 0   1   252:1    9   DRIVE Onln Y 557.861 GB dsbl N N   none -
--------------------------------------------------------------------------

DG=Disk Group Index|Arr=Array Index|Row=Row Index|EID=Enclosure Device ID
DID=Device ID|Type=Drive Type|Onln=Online|Rbld=Rebuild|Dgrd=Degraded
Pdgd=Partially degraded|Offln=Offline|BT=Background Task Active
PDC=PD Cache|PI=Protection Info|SED=Self Encrypting Drive|Frgn=Foreign
DS3=Dimmer Switch 3|dflt=Default|Msng=Missing|FSpace=Free Space Present

Virtual Drives = 1

VD LIST :
=======

-----------------------------------------------------------
DG/VD TYPE State Access Consist Cache sCC       Size Name
-----------------------------------------------------------
0/0   RAID1 Dgrd RW     Yes     NRWBD -   557.861 GB     <<<<<<<<<<<<<<<<<<<<<<<<<<< you will see that the virtual disk is in a degraded mode
-----------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|B=Blocked|Consist=Consistent|
R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

Physical Drives = 2

PD LIST :
=======

-------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model            Sp
-------------------------------------------------------------------------
252:0     8 Dgrd   0 557.861 GB SAS HDD N   N 512B H109060SESUN600G U
252:1     9 Onln   0 557.861 GB SAS HDD N   N 512B H109060SESUN600G U
-------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

Cachevault_Info :
===============

------------------------------------
Model State   Temp Mode MfgDate
------------------------------------
CVPM02 Optimal 31C -    2014/12/26
------------------------------------

2. Now that you've determined which disk drives needs to be replaced, use the storcli64 command to flash the amber led on the disk to better identify which disk needs to be pulled. From the example above, disk 0 needs to be replaced, use this command:

# /opt/MegaRAID/storcli/storcli64 /c0/e252/s0 start locate
Controller = 0
Status = Success
Description = Start Drive Locate Succeeded.

3. Pull the disk and replace it with a new one.

4. The new disk should now be in a rebuild status:

# /opt/MegaRAID/storcli/storcli64 /c0 show
Generating detailed summary of the adapter, it may take a while to complete.

Controller = 0
Status = Success
Description = None

Product Name = LSI MegaRAID 9361-8i
Serial Number = SV50453013
SAS Address = 500605b00a08c7f0
PCI Address = 00:23:00:00
System Time = 03/19/2015 23:05:46
Mfg. Date = 01/24/15
Controller Time = 03/20/2015 05:05:54
FW Package Build = 24.3.0-0063
BIOS Version = 6.17.04.2_4.16.08.00_0x06060A05
FW Version = 4.230.40-3739
Driver Name = megaraid_sas
Driver Version = 06.803.02.00-rc1
Vendor Id = 0x1000
Device Id = 0x5D
SubVendor Id = 0x1000
SubDevice Id = 0x9361
Host Interface = PCIE
Device Interface = SAS-12G
Bus Number = 35
Device Number = 0
Function Number = 0
Drive Groups = 1

TOPOLOGY :
========

--------------------------------------------------------------------------
DG Arr Row EID:Slot DID Type State BT       Size PDC PI SED DS3 FSpace
--------------------------------------------------------------------------
0 -   -   -        -   RAID1 Dgrd N 557.861 GB dsbl N N   none N
0 0   -   -        -   RAID1 Dgrd N 557.861 GB dsbl N N   none N
0 0   0   252:0    10 DRIVE Rbld Y 557.861 GB dsbl N N   none -   <<<<<<<<<<<<<<< This disk is now syncing with the other mirror
0 0   1   252:1    9   DRIVE Onln N 557.861 GB dsbl N N   none -
--------------------------------------------------------------------------

DG=Disk Group Index|Arr=Array Index|Row=Row Index|EID=Enclosure Device ID
DID=Device ID|Type=Drive Type|Onln=Online|Rbld=Rebuild|Dgrd=Degraded
Pdgd=Partially degraded|Offln=Offline|BT=Background Task Active
PDC=PD Cache|PI=Protection Info|SED=Self Encrypting Drive|Frgn=Foreign
DS3=Dimmer Switch 3|dflt=Default|Msng=Missing|FSpace=Free Space Present

Virtual Drives = 1

VD LIST :
=======

-----------------------------------------------------------
DG/VD TYPE State Access Consist Cache sCC       Size Name
-----------------------------------------------------------
0/0   RAID1 Dgrd RW     No      NRWBD -   557.861 GB
-----------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|B=Blocked|Consist=Consistent|
R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency

Physical Drives = 2

PD LIST :
=======

-------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model            Sp
-------------------------------------------------------------------------
252:0    10 Rbld   0 557.861 GB SAS HDD N   N 512B H109060SESUN600G U
252:1     9 Onln   0 557.861 GB SAS HDD N   N 512B H109060SESUN600G U
-------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

Cachevault_Info :
===============

------------------------------------
Model State   Temp Mode MfgDate
------------------------------------
CVPM02 Optimal 33C -    2014/12/26
------------------------------------

5. To view the progress of the rebuild status, use the following command:

# /opt/MegaRAID/storcli/storcli64 /c0/e252/s0 show rebuild
Controller = 0
Status = Success
Description = Show Drive Rebuild Status Succeeded.

------------------------------------------------------
Drive-ID Progress% Status Estimated Time Left
------------------------------------------------------
/c0/e252/s1 8 In progress 50 Minutes
------------------------------------------------------

Note * If the new disk does not start to rebuild automatically, here are a few things to check -

Is the foreign flag set on the disk?

To check if there is a foreign flag on the disk, use this command:

# /opt/MegaRAID/storcli/storcli64 /c0/fall show

if the disk shows a foreign flag, clear it:

# /opt/MegaRAID/storcli/storcli64 /c0/fall clear

then start the rebuild:

# /opt/MegaRAID/storcli/storcli64 /c0/e252/s0 start rebuild

If the disk in Unconfigured Bad status (UBad)?

This sometimes happen when you unplug/plug the same disk into the same slot.

If this is a known good disk, first change the status to Unconfigured Good (UGood):

# /opt/MegaRAID/storcli/storcli64 /c0/e252/s0 set good

then start the rebuild:

# /opt/MegaRAID/storcli/storcli64 /c0/e252/s0 start rebuild

6. If you used the storcli command to flash the amber led on the disk drive, you will need to turn it off:

# /opt/MegaRAID/storcli/storcli64 /c0/e252/s1 stop locate
Controller = 0
Status = Success
Description = Stop Drive Locate Succeeded.

/c0/e252/s0 start rebuild

7. Updating disk firmware if needed

If the firmware on the disk is older than what the ODA software image requires, then you will need to update the firmware on the disk.

To upgrade the disk firmware, use the oakcli update -patch command, for example:

# oakcli update -patch 12.1.2.6.0 --infra <<<<<<<<<<<<<< be sure to use the version of your ODA software
For version 12.1.2.7.0 and above, the command syntax has changed to --storage:

# oakcli update -patch 12.1.2.7.0 --storage

For systems running the dcs stack, use the following command:

# /opt/oracle/dcs/bin/odacli update-storage -v 12.2.1.2.0

References

<BUG:15789419> - SUNBT7166073 REPLACED DISKS ASSIOCIATED WITH OLD JBOD SERIAL NUMBER
https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:DE2-24C+and+DE2-24P+Serial+number+re-programming+procedure.
<NOTE:1920586.1> - Oracle ZFS Storage Appliance: DE2 Tray Serial Number Reprogramming

Attachments

This solution has no attachment