Exadata X7-2 Hardware
 
Zero Data Loss Recovery Appliance X7 Hardware
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

In this Document

Goal

Solution

References

Oracle Confidential PARTNER - Available to partners (SUN).
Reason: Exadata internal only for Oracle support engineers use and approved HW partners

Applies to:

Goal

How to Replace an Exadata X7-2 Compute Node Server NIC 10GbE/25GbE PCIE Card.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED:
Exadata X7-2 Training

TIME ESTIMATE: 60 minutes

TASK COMPLEXITY: 2

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS

PROBLEM OVERVIEW: An Exadata X7-2 Compute Node Server NIC 10GbE/25GbE PCIE card needs replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY?:

IMPORTANT NOTE TO TSC ENGINEER: CUT & PASTE the “CUSTOMER ACTIVITY” sections of the Pre-Replacement and Post-Replacement steps into a SR Note and ensure the customer is aware to do these steps prior to the scheduled field engineer activity, and during and after the replacement activity.

CUSTOMER ACTIVITY:

Shutdown of the database node is required prior to the part replacement:

If running Linux or Solaris native - follow Steps 1 to 9 of MOS Note:
How to shutdown the Exadata database nodes and storage cells in a rolling fashion so certain hardware tasks can be performed. (Doc ID 1539451.1)

If running OVM then follow MOS Note:
How to Shutdown and Startup Exadata database nodes running OVM (Doc ID 2367609.1)

WHAT ACTION DOES THE ENGINEER/ADMINISTRATOR NEED TO TAKE?:

Prepare the Server for Service

The customer should have already prepared the server and powered it off. If not, provide them the instructions in the previous section.

1. Extend the server to the maintenance position
2. Disconnect the power cords from the power supplies
3. Attach an anti-static wrist strap to your wrist and to a metal area on the chassis or the rack.
4. Remove the server top cover. Use a Torx T10 screwdriver to unlock the release button latch.

Removing the NIC 10GbE/25GbE PCIE card

The 10Gb/25Gb PCIe card NIC is located in Slot 1 or Slot 3 depending on the Compute Node configuration.

1. Identify the PCIe card to be replaced and disconnect any external cables from the card. Note the cable locations and their order to ensure they are reinstalled in their proper locations.  

2. Remove the transceivers from the PCIe NIC card slots and place on an anti-static mat.

3. For the card installed in Slot 1:

        a. Lift the green-tabbed latch on the rear of the server's chassis next to the PCIe Slot 1 to release the PCIe card's rear bracket.
        b. Lift the riser release lever with one hand and use your other hand to remove the riser from the motherboard
        c. Place the riser and card on an anti-static mat.
        d. Remove the PCIe card from the PCIe riser. Hold the riser in one hand and use your other hand to carefully pull the PCIe card connector from the riser.
        e. Disconnect the rear bracket that is attached to the PCIe card from the rear of the PCIe riser.
        f. Place the PCIe card on an antistatic mat.

   For the card installed in Slot 3:

        a. Open the green-tabbed latch located on the rear of the server's chassis next to the PCIe slot 3 to release the PCIe card holding bracket.
        b. To release the riser from the motherboard connector, lift the riser's green-tabbed release lever to the open position.
        c. Grasp the riser with both hands and remove it from the server carefully.
        d. Remove the Slot 3 NIC card from the riser carefully such as to not pull out the internal SAS HBA and cables to it:
              i. Hold the riser in one hand and use your other hand to carefully pull the PCIe card connector from the riser. 
              ii. Disconnect the rear bracket that is attached to the PCIe card from the rear of the PCIe riser.
              iii. Place the NIC card on an anti-static mat
        e. Re-insert the riser back into the server slot temporarily such that the cables to the internal SAS HBA do not need to be removed. 

Installing the NIC 10GbE/25GbE PCIE card

1. Remove the replacement PCIe card from it's anti-static bag and place on an anti-static mat.

2. For the card installed in Slot 1:

       a. Insert the rear bracket that is attached to the PCIe card into the PCIe riser.
       b. Hold the riser in one hand and use your other hand to carefully insert the PCIe card connector into the Riser.
       c. Raise the Slot 1 PCIe riser release lever (marked with a green tab) to the open (up) position
       d. Gently press the riser into the motherboard connector until it seats and press the green-tabbed, riser release lever to the closed (down) position.
       e. Close the green-tabbed latch on the rear of the server's chassis to secure the PCIe card's rear bracket to the server's chassis.

   For the card installed in Slot 3:

       a. Lift the riser out of the server carefully such that the cables to the internal SAS HBA do not need to be removed
       b. Insert the rear bracket that is attached to the PCIe card into the PCIe riser.
       c. Hold the riser in one hand and use your other hand to carefully insert the PCIe card connector into the Riser.
       d. Raise the green-tabbed release lever on the PCIe riser to the open (up) position, and then gently press the riser into the motherboard connector until it is seated.
       e. Ensure that the rear bracket on the internal SAS HBA card in PCIe slot 4 is connected to the slot in the server's chassis side wall. If the bracket is not connected, remove the riser and re-position it so that the rear bracket connects to the side wall, then gently press the riser into the motherboard connector.
       f. Press the green-tabbed release lever on the PCIe riser to the closed (down) position.
       g. To secure the PCIe card's rear bracket to the server, close the green-tabbed latch on the rear of the server's chassis.
       
3. Re-install the transceivers into the replacement PCIe NIC card slots

4. Reconnect any external cables that were previously disconnected to the replacement PCIe card, ensuring they go back into the correct original locations.


Return the Server to Operation

1. Install the server top cover. Use a Torx T10 screwdriver to lock the release button latch.
2. Reconnect the power cords to the server power supply and connect any other cables to their original locations.
3. Return the server to the normal rack position.
4. Once the power cords have been re-attached and the ILOM has booted you will see a slow blink on the green LED for the server. Power on the server by pressing the power button on the front of the unit.
5. Connect to the server console via the ILOM and monitor the boot.
    By default the ILOM serial console displays the primary console output.
    In the event of unexpected boot behavior, it is advisable to connect to both ILOM serial and ILOM graphics consoles at the same time and monitor.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE FIELD ENGINEER/ADMINISTRATOR NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE?:

FIELD SERVICE ENGINEER and CUSTOMER ACTIVITY:

1. Verify all expected hardware is visible to the server and the fault is cleared. Assistance from the customer for server login access will be required.

2. Verify there are no outstanding faults in ILOM:

3. Verify there are no outstanding alerts in the Database Node:

4. Verify the network ports are linked. For the 10Gb/25Gb PCIe card NIC this will be eth3 and eth4, and one of the bond interfaces (0 or 1) should be up and "link detected" is "Yes"; adjust the command according to the output:

5. Re-enable and restart the Database services:

If running Linux or Solaris native - follow Steps 11 to 14 of MOS Note:
How to shutdown the Exadata database nodes and storage cells in a rolling fashion so certain hardware tasks can be performed. (Doc ID 1539451.1)

If running OVM then follow MOS Note:
How to Shutdown and Startup Exadata compute nodes running OVM (Doc ID 2367609.1)

PARTS NOTE:

7339763 [F] Dual 10/25-Gigabit SFP28 Ethernet

REFERENCE INFORMATION:

Oracle Exadata Database Machine Maintenance Guide: https://docs.oracle.com/cd/E80920_01/DBMMN/maintaining-exadata-database-servers.htm#DBMMN22020

Oracle Server X7-2 Documentation https://docs.oracle.com/cd/E72435_01/index.html

How to shutdown the Exadata database nodes and storage cells in a rolling fashion so certain hardware tasks can be performed. (Doc ID 1539451.1)

How to Shutdown and Startup Exadata compute nodes running OVM (Doc ID 2367609.1)

This solution has no attachment