Big Data Appliance X3-2 Hardware
 
Big Data Appliance X3-2 Full Rack
 
Big Data Appliance X3-2 In-Rack Expansion
 
Big Data Appliance X4-2 Hardware
 
Big Data Appliance X4-2 Full Rack
 
Big Data Appliance Hardware
 
Big Data Appliance X4-2 Starter Rack
 
Big Data Appliance X4-2 In-Rack Expansion
 
Big Data Appliance X3-2 Starter Rack
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

In this Document

Goal

Solution

References

Oracle Confidential PARTNER - Available to partners (SUN).
Reason: FRU CAP document

Applies to:

Goal

 How to Replace a Big Data Appliance (Original V1, X3-2 or X4-2) Faulty RAID HBA

Solution

DISPATCH INSTRUCTIONS:
- WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED: BDA Trained
- TIME ESTIMATE: 90 Minutes
- TASK COMPLEXITY: 3

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

- PROBLEM OVERVIEW: A faulty RAID HBA in a Big Data Appliance server node has been diagnosed as needing replacement.

- WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY?:

The instructions below assume the Customer system administrator is available and working with the field engineer onsite to manage the host OS and BDA services.
They are provided here to allow the FE to have all the available steps needed when onsite, and can be done by the FE if the customer system administrator wants or allows or needs help with these steps.
The server that contains the faulty RAID HBA card should have its services offlined and system powered off.

Preparation for replacement:

1. If the OS is still operating and available, then revert all the RAID disk volumes to WriteThrough mode to ensure all data in the RAID cache memory is flushed to disk and not lost when replacement of the HBA occurs. Set all logical volumes cache policy to WriteThrough cache mode:

Verify the current cache policy for all logical volumes is now WriteThrough :

2. The Customer’s system administrator should shutdown the server node and BDA services following the shutdown instructions for Big Data Appliance detailed in MOS Note 2099858.1

- WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE:

Physical RAID HBA replacement:

1. Slide out the server for maintenance.

Do not remove any cables prior to sliding the server forward, or the loose cable ends will jam in the cable management arms. Take care to ensure the cables and Cable Management Arm is moving properly. Refer to Note 1444683.1 for CMA handling training.

2. Disconnect the AC power cords.

3. Unlatch and slide off the top cover of the server.

4. Remove the old RAID HBA.

BDA (V1) Server Nodes:
These steps are relevant to BDA nodes based on Sun Fire x4270 M2 Server.
   a) Remove the IB cables from the IB card in slot 3 below the HBA making a note of which port each cable goes into so they can go back into the same port.
   b) Remove back panel PCI cross bar
      i) Loosen the two captive Phillips screws on each end of the crossbar
      ii) Lift the PCI crossbar up and back to remove it from the chassis
   c) Remove the PCIe Riser 2 containing the RAID HBA card to be serviced
      i) Loosen the captive screw holding the riser to the motherboard
      ii) Lift up the riser and the PCIe cards that are attached to it as a unit.
   d) Disconnect the SAS cables from the RAID HBA card making a note of which port each cable goes into so they can go back into the same port.
   e) Extract the RAID HBA card from the PCIe Riser assembly, and place on an anti-static mat.

BDA X3-2 and X4-2 Server Nodes:
These steps are relevant to BDA nodes based on Sun Server X3-2L and Sun Server X4-2L.
   a) Rotate the PCIe card slot 6 locking mechanism latch out to disengage the RAID HBA card that has failed.
   b) Lift up and remove the RAID HBA card from the server.
   c) Disconnect the SAS cables from the RAID HBA card making a note of which port each cable goes into so they can go back into the same port.
   d) Place the removed RAID HBA on an anti-static mat.

5. Remove the RAID HBA's battery from the old RAID HBA

BDA (V1) Server Nodes:
These steps are relevant to BDA nodes based on Sun Fire x4270 M2 Server.
   a) Use a No. 1 Phillips screwdriver to remove the 3 retaining screws that secure the battery to the HBA from the underside of the card. Do not attempt to remove any screws from the top side of the HBA.
   b) Detach the battery pack including circuit board from the HBA by gently lifting it from its circuit board connector on the top side of the HBA

BDA X3-2 and X4-2 Server Nodes:
If the node is Sun Server X3-2L and does not have the remote-mounted battery kit installed, then follow the steps for BDA (V1) Server Nodes.
These steps are relevant to BDA nodes based on Sun Server X3-2L and Sun Server X4-2L with the remote-mounted battery kit installed.
   a) Use a No. 1 Philips screwdriver to remove the 3 retaining screws that secure the interface board to the HBA from the underside of the card.
   b) Detach the HBA Interface board from the HBA by gently lifting it from its circuit board connector on the top side of the HBA.
   c) Keep the cable and cable routing guide connected to the HBA interface board, and place on an anti-static mat.

6. Reinstall the HBA's battery or remote-mounted battery HBA Interface Board onto the new HBA. Reverse the removal instructions in step 5.
Note: If the system has a remote-mounted battery, take care to route the cable correctly. See MOS Note 1561949.1 for more guidance if necessary, including photos and videos.

7. Install the new RAID HBA PCI Card.
    On BDA (V1) Server nodes based on Sun Fire X4270M2, this will be into the empty slot on PCIe Riser 2.
    On BDA X3-2 and X4-2 Server nodes this will be into PCIe Slot 6.

Reverse the removal instructions in Step 4, taking care to get the SAS cables re-connected to the same ports they were removed from. If reversed, this may affect disk slot mappings.
Note: On BDA (V1) Server nodes based on Sun Fire X4270M2 server, take care to also put the IB cables back into the original ports, as well, in the correct orientation. IB cables are factory labeled with the port identification where port 2 is the port nearest the PCI connector, and port 1 is the port near the top side of the card. The cables should be inserted with the latch release tab on the down side, so they fully seat and latch. If inserted upside down, they will not fully seat or latch.

8. Install the top cover

9. Install the AC power cords

10. Slide the server back into the rack.

OBTAIN CUSTOMER ACCEPTANCE

- WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

1. Once the ILOM has booted you will see a slow blink on the green LED for the server. Press the power button on the front of the server to power on the unit.

2. During boot, monitor the graphics console through either ILOM javaconsole or the local KVM. When loading its BIOS ROM, the new RAID controller will detect the RAID configuration on the disks and complain it has a Foreign configuration. This is expected. At the prompt, press "F" or "C" to accept the foreign configuration or enter the controller BIOS utility. If you press any other key to continue, then the controller will not import the RAID and will fail to find a bootable disk. If this occurs, it is safe to press "ctrl-alt-del" and reset and get the "F" or "C" prompt again.
Press F or C
Press C
Press Y

2. When the utility loads, there should only be 1 adapter. Select the "Start" button.

3. The foreign configuration screen is shown. Select "Configuration" from the drop down, and select the "Preview" button.

4. Verify the configuration looks correct on the "Virtual Drives" side and select the "Import" button if it is. The correct configuration should be 12 RAID0's, 1 per disk.

5. This will bring you back to the Logical View screen where the virtual drives should be listed out on the right side. Select the "Exit" link from the left side menu. This will bring you to the "Please Reboot" screen. Press "Ctrl-Alt-Del" to reboot the machine and boot the OS.

6. After the OS has booted, login to the OS with ‘root’ privilege.

7. Run the following to update the RAID HBA to the correct supported firmware for the image:

After the firmware updates, the server will reboot again. The disk volumes should remain intact and boot up to the OS again.

8. After the OS is up, login as root and validate the physical and logical volumes are seen properly from the new RAID HBA in the OS and that the battery is seen:
The following command should show 12 disks:

If the device count is not correct check also that the LSI controller has the correct Virtual Drives configured and in Optimal state, physically Online and spun up, with no Foreign configuration. There should be Virtual Drives 0 to 11, and the physical slots 0 to 11 should be allocated to 1 each (not necessarily the same 0:0 1:1 etc. mapping).

If this is not correct, then there is a problem with the disk volumes that may need additional assistance to correct. The server should be re-opened and the device connections and boards checked to be sure they are secure and well seated BEFORE the following commands are issued.

9. Set all logical drives cache policy to WriteBack cache mode:

Verify the current cache policy for all logical drives is now using WriteBack cache mode:

10. On BDA (V1) systems based on Sun Fire X4270M2 server, verify also the InfiniBand links are up at 40Gbps as the cables were disconnected:

11. Once the hardware is verified as up and running, the Customer's system administrator will need to verify the BDA services are up following the startup procedures for Big Data Appliance detailed in MOS Note 2099858.1

PARTS NOTE:
375-3701 or 7047503 are the same, difference is RoHS2013 compliance.
8-Port 6Gbps SAS-2 RAID PCI Express HBA, B4 ASIC

References

This solution has no attachment