How to Replace an Exalogic Elastic Cloud X2-2 Compute node Raid Controller

Asset ID:	1-71-1401698.1
Update Date:	2016-01-20
Keywords:

Solution Type Technical Instruction Sure

Solution 1401698.1 : How to Replace an Exalogic Elastic Cloud X2-2 Compute node Raid Controller

Applies to:

Oracle Exalogic Elastic Cloud X2-2 Half Rack - Version X2 and later
Oracle Exalogic Elastic Cloud X2-2 One-Eighth Rack - Version X2 and later
Oracle Exalogic Elastic Cloud X2-2 Hardware - Version X2 and later
Oracle Exalogic Elastic Cloud X2-2 Qtr Rack - Version X2 and later
Oracle Exalogic Elastic Cloud X2-2 Full Rack - Version X2 and later
Information in this document applies to any platform.

Goal

How to Remove and Replace a Failed Raid Controller on a Exalogic Compute Node?

Solution

DISPATCH INSTRUCTIONS
- WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED:

The FSE needs to be Exalogic Trained.

The FSE should work closely with the administrator to also ensure any pre or post work is completed.

- TIME ESTIMATE: 60 minutes
- TASK COMPLEXITY: 3

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:
- PROBLEM OVERVIEW:
- WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE
RESOLUTION ACTIVITY?:

NOTE - If this is an Engineered system then the system administrator should prepare the system for service by performing any application related functions required to shutdown the node. This might include but is not limited to performing a system backup, failover of application or services, and finally a system shutdown.

It is highly recommended to make a backup of all disk partitions prior to RAID HBA replacement.

Note: If this is an Exalogic server running OVS with virtual servers, please verify customer has readied the server by moving VM's off of the server and placed the server in maintenance mode. See Document: Making an OVS Node Unavailable for vServer Placement in Exalogic Virtual Environments using EMOC (Doc ID 1551724.1)

- WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE:

Remove the old HBA PCI Card

1. On Storage cells remove the IB cables from the IB card in slot 3 above the HBA.

2. Remove back panel PCI cross bar

a) Loosen the two captive Phillips screws on each end of the crossbar
b) Lift the PCI crossbar up and back to remove it from the chassis

3. Remove the PCI Riser containing the PCI card to be serviced

a) Loosen the captive screw holding the riser to the motherboard
b) Lift up the riser and the PCI card that is attached to it as a unit.

4. Disconnect the SAS cables from PCI card making a note of which port each cable goes into.

5. Extract the RAID HBA card from the PCI riser assembly

Remove the HBA's battery from the old HBA

1. Use a No. 1 Phillips screwdriver to remove the 3 retaining screws that secure the battery to the HBA from the underside of the card. Do not attempt to remove any screws from the top side of the HBA.

2. Detach the battery pack including circuit board from the HBA by gently lifting it from its circuit board connector on the top side of the HBA

Reinstall the HBA's battery onto the new HBA

Reverse the removal instructions

Install the new HBA PCI Card

Reverse the removal instructions, taking care to get the cables re-connected to the same ports they were removed from. If reversed, this may affect disk slot mappings.

OBTAIN CUSTOMER ACCEPTANCE
- WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Post-Replacement RAID Card additional steps:

Power on Compute Node:

1. Once the power cords have been re-attached, slide the server back into the rack.
2. Once the ILOM has booted you will see a slow blink on the green LED for the server. Power on the server by pressing the power button on the front of the unit.

Accepting the Foreign Configuration:

1. During boot, monitor the graphics console through ILOM javaconsole. When loading its BIOS ROM, the new RAID controller will detect the RAID configuration on the disks and complain it has a Foreign configuration. This is expected. At the prompt, press 'F' or 'C' to accept the foreign configuration or enter the controller BIOS utility. If you press any other key to continue, then the controller will not import the RAID and will fail to find a bootable disk. If this occurs, it is safe to press 'ctrl-alt-del' and reset and get the 'F' or 'C' prompt again.
2. When the utility loads, there should only be 1 adapter. Select the 'Start' button.
3. The foreign configuration screen is shown. Select 'Configuration 1' from the drop down, and select the 'Preview' button.
4. Verify the configuration looks correct on the 'Virtual Drives' side and select the 'Import' button if it is. The correct configuration should be as follows:

Compute Nodes will have a single Raid1 Virtual Drive with SSD disk slot 0 & 1.

NOTE: If the foreign configuration fails to import, as it may if the firmware on the replacement is different from the firmware on the failed HBA, then you may need to recreate the volume. If this is the case, please escalate to the TSC for assistance with this process.

5. This will bring you back to the Logical View screen where the virtual drives should be listed out on the right side. Select the 'Exit' link from the left side menu. This will bring you to the 'Please Reboot' screen. Press 'Ctrl-Alt-Del' to reboot the machine.

Additional Checks:

- CheckHWnFWProfile, verify Raid Controller firmware is current.

# /opt/exalogic.tool/tools/CheckHWnFWProfile -F
Verifying Firmware...
Supported BIOS Version: 08060108-12/27/2010
Current BIOS Version : 08060108-12/27/2010
BIOS is at the supported version

Supported ILOM Version: 3.0.14.11.b r62978
Current ILOM Version : 3.0.14.11.b r62978
ILOM is at the supported version.

Supported OFED Version: BXOFED-1.5.2-1.3.8000
Current OFED Version : BXOFED-1.5.2-1.3.8000
OFED is at the supported version.

Supported Infiniband Firmware Version: 2.7.8130
Current Infiniband Firmware Version : 2.7.8130
Infiniband Firmware is at the supported version.

Supported disk controller Version: 12.12.0-0048
Current disk controller Version : 12.12.0-0048
Disk controller is at the supported firmware version.

Note: The firmware should be automatically updated if needed during the boot-up process. If this does not happen you will need to update the firmware manually, see the next section.

Update Raid Controller firmware:

You can perform this update in one of two following methods.

Update firmware using 'UpdateFirmware' command and firmware file located in '/opt/exalogic/firmware/disk_controller/XXXXXX.rom' (in our case 12_12_0_0048.rom ).

# /opt/exalogic.tools/tools/UpdateFirmware -lf /opt/exalogic/firmware/disk_controller/12_12_0_0048.rom

Note: If this operation fails you might want to try the alternate method using 'MegaCli64'.

Update firmware using 'MegaCli64' command and firmware file located in '/opt/exalogic/firmware/disk_controller/XXXXXXXX.rom' (in our case 12_12_0_0048.rom ).

# /opt/MegaRAID/MegaCli/MegaCli64 -adpfwflash -f /opt/exalogic/firmware/disk_controller/12_12_0_0048.rom -noverchk -a0 -silent

PARTS NOTE:

https://support.us.oracle.com/handbook_internal/Systems/Exalogic_X2_2/components.html#SCSI

REFERENCE INFORMATION:

Exalogic Machine Owner's Guide: https://docs.oracle.com/cd/E18476_01/index.htm

X4170M2 Documentation Set: https://docs.oracle.com/cd/E19762-01/index.html

X4170M2 Service Guide: https://docs.oracle.com/cd/E19762-01/E22369-02/E22369-02.pdf

Attachments

This solution has no attachment