Exadata X7-2 Hardware
 
Exadata X7-8 Hardware
 
Oracle SuperCluster M8 Hardware
 
Zero Data Loss Recovery Appliance X7 Hardware
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

In this Document

Goal

Solution

References

Oracle Confidential PARTNER - Available to partners (SUN).
Reason: Exadata internal only for Oracle support engineers use and approved HW partners

Applies to:

Goal

How to Replace an Exadata X7-2 Storage Cell Server Internal RAID HBA Card.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED?:

Exadata X7-2 Training.

TIME ESTIMATE: 60 Minutes

TASK COMPLEXITY: 3

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

PROBLEM OVERVIEW: An Exadata X7-2 Storage Cell Server RAID HBA (SAS disk controller) needs replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

IMPORTANT NOTE TO TSC ENGINEER: CUT & PASTE the “CUSTOMER ACTIVITY” sections of the Pre-Replacement and Post-Replacement steps into a SR Note and ensure the customer is aware to do these steps prior to the scheduled field engineer activity, and during and after the replacement activity.

CUSTOMER ACTIVITY:

Offlining the disk cache and shutdown of the storage cell is required prior to the part replacement.

1. Complete Steps 1 to 5 of MOS Note ID 1188080.1 “Steps to shut down or reboot an Exadata storage cell without affecting ASM”

Where noted, the SQL, CellCLI and commands under ‘root’ should be run by the Customers DBA, unless the Customer provides login access to the Field Engineer.

These steps are also provided in the documentation:
https://docs.oracle.com/cd/E80920_01/DBMMN/maintaining-exadata-storage-servers.htm#DBMMN21128

2. Revert all the RAID disk volumes to WriteThrough mode to ensure all data in the RAID cache memory is flushed to disk and not lost when replacement of the RAID HBA occurs. As 'root' user, set all logical volumes cache policy to WriteThrough cache mode:

3. Verify the current cache policy for all logical volumes is now WriteThrough:

In the volume table, the "Cache" column should report as "NRWTD" where WT indicates WriteThrough.

4. Once all disks are offline and inactive, the customer may shutdown the Cell using the following command:

WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE?:

Prepare the Server for Service

The customer should have already prepared the server and powered it off.  If not, provide them the instructions in the previous section.

1. Extend the server to the maintenance position
2. Disconnect the power cords from the power supplies.
3. Attach an anti-static wrist strap to your wrist and to a metal area on the chassis or the rack.
4. Remove the server top cover. Use a Torx T10 screwdriver to unlock the release button latch.

Removing the RAID HBA

1. The RAID HBA is located in PCIe Slot 11, on the far left side of the chassis when looking from the front. Rotate the Slot 11 PCIe card locking mechanism open.

2. Lift and remove the RAID HBA out of the motherboard slot.

3. Disconnect the 3 SAS cables and the Super Capacitor cable from the RAID HBA card.

4. Place the RAID HBA card on an anti-static mat.

Installing the RAID HBA

1. Unpack the replacement RAID HBA card and place it on an anti-static mat.

2. Connect the Super Capacitor cable to the RAID HBA., and then reconnect the SAS cables that you unplugged during the removal procedure.

3. Connect the 3 SAS Cables to the RAID HBA. Plug each cable into its SAS connector until you hear an audible click.

Ensure the cables are installed in the correct HBA connector slots as they directly connect to the disk slots being connected through them:
 - Cable 1 goes to disk slots 0-3
 - Cable 2 goes to disk slots 4-7
 - Cable 3 goes to disk slots 8-11
 - Disk slots 12-15 connector should be empty.

4. Insert the RAID HBA card into PCIe Slot 11

5. Rotate the PCIe locking mechanism to secure the PCIe HBA card in place. You will hear an audible click when the PCIe card is secured into the slot.

Return the Server to Operation

1. Install the server top cover. Use a Torx T10 screwdriver to lock the release button latch.
2. Reconnect the power cords to the server power supply and connect any other cables to their original locations.
3. Return the server to the normal rack position.
4. Once the power cords have been re-attached and the ILOM has booted you will see a slow blink on the green LED for the server. Power on the
server by pressing the power button on the front of the unit.
5. Connect to the server console via the ILOM and monitor the boot.
   By default the ILOM serial console displays the primary console output.
   In the event of unexpected boot behavior, it is advisable to connect to both ILOM serial and ILOM graphics consoles at the same time and monitor.
6. Enter the BIOS Setup Utility and access the RAID HBA utility.  Import the foreign configuration setup to take the RAID information from the disks, and overwrite any configuration in the HBA.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE?:

FIELD SERVICE ENGINEER and CUSTOMER ACTIVITY:

1. Verify all expected hardware is visible to the server and the fault is cleared. Assistance from the customer for server login access will be required.

2. Verify all the expected disk devices are present. For a 1/8th rack Storage Cell there will be 6 disks, for all others there will be 12 disks:

3. Verify the status of the Super Capacitor is visible and 'Optimal':

4. Set all logical drives cache policy to WriteBack cache mode:

5. Verify all expected logical drives are present and state 'Optl' (Optimal). For 1/8th rack Storage Cells there will be 6 disks, for all others there will be 12 disks:

In the volume table, the "Cache" column should report as "NRWBD" where WB indicates WriteBack.

6. Verify there are no outstanding alerts in the Cell:

7. Re-activate the Storage Cell grid disks. Follow Steps 7 to 10 of Note ID 1188080.1 “Steps to shut down or reboot an Exadata storage cell without affecting ASM”.

These steps are also provided in the documentation:
 https://docs.oracle.com/cd/E80920_01/DBMMN/maintaining-exadata-storage-servers.htm#DBMMN21128

PARTS NOTE:

7332895 [F] 16-Port 12Gbps SAS-3 Internal RAID HBA

REFERENCE INFORMATION:

Oracle Server X7-2L Documentation: https://docs.oracle.com/cd/E72463_01/index.html

Steps to shut down or reboot an Exadata storage cell without affecting ASM (Doc ID 1188080.1)

For a documentation reference, in the "Exadata Database Maintenance Guide", use the section titled "General Maintenance Information" section “Powering On and Off Oracle Exadata Rack/Non-emergency Power Procedures” sub-section “Powering Off (or On) Oracle Exadata Rack/Powering Off Storage Servers.
https://docs.oracle.com/cd/E80920_01/DBMMN/exadata-general-maintenance.htm#GUID-7C50CD88-81E1-479C-A2E4-C68B5618FF7E

This solution has no attachment