Oracle Database Appliance X4-2
 
Oracle Database Appliance X3-2
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

Applies to:

Goal

How to Replace a PCIe Card in slot 1.

On an ODA X3-2, slot 1 contains either a Dual 10-Gigabit Ethernet SFP+ LP (Niantic card) or a Dual 10-Gigabit Ethernet Base-T Gen2 (Twinville card).

On an ODA X4-2, slot 1 contains a Dual 10-Gigabit Ethernet SFP+ LP (Niantic card).

This is an appliance and no other cards should be present in slot 1 of these systems.  Slots 2 and 3 both have a Dual 10-Gigabit Ethernet Base-T Gen2 (Erie Ext) card.  The procedure to replace the Erie cards is in Doc ID 2018427.1

Solution

CAP PROBLEM OVERVIEW: PCIe Card Replacement

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE ENGINEER NEED:
Oracle Database Appliance Training

TIME ESTIMATE: 60 minutes

TASK COMPLEXITY: 0-CRU

FIELD ENGINEER INSTRUCTIONS

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

If the system is still up and functioning, customer should perform an orderly and graceful shutdown of applications and OS.  Then power off the server and remove the AC power cords from the system.

A data backup is not a prerequisite but is a wise precaution.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

1. Prepare the server for service.

1. Power off the server and disconnect the power cords from the power supplies.
2. Extend the server to the maintenance position in the rack.
3. Attach an anti-static wrist strap.
4. Remove the top cover.

2. Locate and Remove the PCIe card.

1. There are three external and one internal PCIe slots in the system. They are numbered 1, 2, and 3 from left to right when you view the server from the rear. (The internal PCIe slot 4 is reserved for an internal SAS controller, replacement of this card is detailed in Doc ID 1592317.1)
2. Identify the PCIe card to be replaced and disconnect any external or internal cables from the card. Note the cable locations and their order to ensure they are reinstalled in their proper locations.
3. For cards installed in either the slot 1 or 2 risers:
   
   1. lift the green-tabbed latch on the rear of the server's chassis next to the PCIe slot to release the PCIe card's rear bracket.
   2. Lift the riser release lever with one hand and use your other hand to remove the riser from the motherboard
   3. place the riser and card on an anti-static mat and make a note of the slot in which the PCIe riser was originally installed so that it can be reinstalled into it's proper slot.
4. For cards installed in the Slot 3 and 4 PCIe riser:
   
   1. open the green-tabbed latch located on the rear of the server's chassis next to the PCIe slot 3 to release the PCIe card holding bracket.
   2. To release the riser from the motherboard connector, lift the riser's green-tabbed release lever to the open position.
   3. Slide the plastic PCIe card retainer, which is mounted on the side of the chassis, forward to release the card(s) installed in the riser
   4. Grasp the riser with both hands and remove it from the server
   5. Disconnect the SAS storage drive (HDD) cables from the internal HBA card installed in PCIe slot 4 Note the cable locations and their order to ensure they are reinstalled in their proper locations.
   6. place the riser and cards on an anti-static mat and make a note of the slot in which the PCIe riser was originally installed so that it can be reinstalled into it's proper slot.
5. Remove the PCIe card from the PCIe riser. Hold the riser in one hand and use your other hand to carefully pull the PCIe card connector out of the riser.
6. Disconnect the rear bracket that is attached to the PCIe card from the rear of the PCIe riser.
7. Place the PCIe card on an antistatic mat.

3. Replace the PCIe card.

1. Remove the replacment PCIe card from it's anti-static bag and place on an anti-static mat.
2. Insert the rear bracket that is attached to the PCIe card into the PCIe riser.
3. Hold the riser in one hand and use your other hand to carefully insert the PCIe card connector into the Riser.
4. Install the PCIe riser with the installed PCIe card into the server.
5. For cards installed in either the slot 1 or 2 risers:
   1. raise the PCIe riser release lever (marked with a green tab) to the open (up) position
   2. Making sure to replace the riser into the same position from which it was removed, gently press the riser into the motherboard connector until it seats and press the green-tabbed, riser release lever to the closed (down) position.
   3. Close the green-tabbed latch on the rear of the server's chassis next to the applicable PCIe slot to secure the PCIe card's rear bracket to the server's chassis.
6. For the PCIe slots 3 and 4 riser,
   1. reconnect the SAS cable(s) to the internal HBA card.Be sure to connect the SAS cable for storage drives 0 through 3 (HDDs 0-3) to the connector that is farther from the riser in which the HBA card is installed.
   2. Raise the PCIe riser green-tabbed release lever to the open (up) position and gently press the riser into the motherboard connector until it seats.
   3. Ensure that the rear bracket on the internal HBA card in PCIe slot 4 is connected to the slot in the server's chassis side wall. If the bracket is not connected, remove the riser and reposition it so that the rear bracket connects to the side wall, then gently press the riser into the motherboard connector.
   4. Slide the plastic PCIe card retainer that is mounted on the side of the chassis toward the back of the server to secure the card(s) installed in the riser.
   5. Press the green-tabbed release lever on the PCIe riser to the closed (down) position.
   6. To secure the PCIe card's rear bracket to the server, close the green-tabbed latch on the rear of the server's chassis.
7. Reconnect any external or internal cables that were previously disconnected to the replacement PCIe card.

4.Return the Server to operation

1. Replace the top cover
2. Remove any anti-static measures that were used.
3. Return the server to it's normal operating position within the rack.
4. Re-install the AC power cords and any data cables that were removed but not yet replaced.
5. Login to the ILOM and check to see if any faults were logged against the PCIe slot. If so clear the fault by logging into the fault managment shell. -  
   > start /SP/faultmgmt/shell
   Are you sure you want to start /SP/faultmgmt/shell (y/n)? y
   
   faultmgmtsp> fmadm faulty
   ...
   FRU : /SYS/MB/PCIE2
   ...
   faultmgmtsp> fmadm repaired /SYS/MB/PCIE2
   faultmgmtsp> fmadm faulty -a
   No faults found
   faultmgmtsp> exit
6. Power on server. Verify that the Power/OK indicator led lights steady on.

 IMPORTANT - the network config files of eth0/eth1 may manually need to edited to reflect the new mac addresses before the network card will work properly again

Below is a sample output of what the network config files look like.  This was taken from an ODA X3-2 node running Bare Metal.  The ODA X4-2 nodes have a fiber card, which will look a little different.  If the nodes are running Virtualized platform, make sure the edits are made in DOM0.

To obtain the mac addresses, you can use ifconfig -a command or ethtool -P eth# command.

Once the edits are done, restart network services:

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Boot up system and verify full functionality

REFERENCE INFORMATION:
Oracle Database Appliance Documentation

http://docs.oracle.com/cd/E22693_01/index.htm

This solution has no attachment