Oracle Database Appliance X3-2
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

Applies to:

Goal

How to Replace a 10-Gigabit Ethernet PCIe Card in an Oracle Database Appliance X3-2

Solution

CAP PROBLEM OVERVIEW: 10-Gigabit Ethernet PCIe Card Replacement

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE ENGINEER NEED:
Sun Server X3-2 Server Training.

TIME ESTIMATE: 60 minutes

TASK COMPLEXITY: 0

FIELD ENGINEER INSTRUCTIONS

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

If the system is still up and functioning, customer should perform an orderly and graceful shutdown of applications and OS.  Then power off the server and remove the AC power cords from the system.

A data backup is not a prerequisite but is a wise precaution.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

1. Prepare the server for service.

1. Power off the server and disconnect the power cords from the power supplies.
2. Extend the server to the maintenance position in the rack.
3. Attach an anti-static wrist strap.
4. Remove the top cover.

2. Locate and Remove the PCIe card.

1. There are three external and one internal PCIe slots in the system. They are numbered 1, 2, and 3 from left to right when you view the server from the rear. (The internal PCIe slot 4 is reserved for an internal SAS controller, replacement of this card is detailed in Doc ID 1501038.1)
2. Identify the PCIe card to be replaced and disconnect any external or internal cables from the card. Note the cable locations and their order to ensure they are reinstalled in their proper locations.
3. For cards installed in either the slot 1 or 2 risers:
   
   1. lift the green-tabbed latch on the rear of the server's chassis next to the PCIe slot to release the PCIe card's rear bracket.
   2. Lift the riser release lever with one hand and use your other hand to remove the riser from the motherboard
   3. place the riser and card on an anti-static mat and make a note of the slot in which the PCIe riser was originally installed so that it can be reinstalled into it's proper slot.
4. For cards installed in the Slot 3 and 4 PCIe riser:
   
   1. open the green-tabbed latch located on the rear of the server's chassis next to the PCIe slot 3 to release the PCIe card holding bracket.
   2. To release the riser from the motherboard connector, lift the riser's green-tabbed release lever to the open position.
   3. Slide the plastic PCIe card retainer, which is mounted on the side of the chassis, forward to release the card(s) installed in the riser
   4. Grasp the riser with both hands and remove it from the server
   5. Disconnect the SAS storage drive (HDD) cables from the internal HBA card installed in PCIe slot 4 Note the cable locations and their order to ensure they are reinstalled in their proper locations.
   6. place the riser and cards on an anti-static mat and make a note of the slot in which the PCIe riser was originally installed so that it can be reinstalled into it's proper slot.
5. Remove the PCIe card from the PCIe riser. Hold the riser in one hand and use your other hand to carefully pull the PCIe card connector out of the riser.
6. Disconnect the rear bracket that is attached to the PCIe card from the rear of the PCIe riser.
7. Place the PCIe card on an antistatic mat.

3. Replace the PCIe card.

1. Remove the replacment PCIe card from it's anti-static bag and place on an anti-static mat.
2. Insert the rear bracket that is attached to the PCIe card into the PCIe riser.
3. Hold the riser in one hand and use your other hand to carefully insert the PCIe card connector into the Riser.
4. Install the PCIe riser with the installed PCIe card into the server.
5. For cards installed in either the slot 1 or 2 risers:
   1. raise the PCIe riser release lever (marked with a green tab) to the open (up) position
   2. Making sure to replace the riser into the same position from which it was removed, gently press the riser into the motherboard connector until it seats and press the green-tabbed, riser release lever to the closed (down) position.
   3. Close the green-tabbed latch on the rear of the server's chassis next to the applicable PCIe slot to secure the PCIe card's rear bracket to the server's chassis.
6. For the PCIe slots 3 and 4 riser,
   1. reconnect the SAS cable(s) to the internal HBA card.Be sure to connect the SAS cable for storage drives 0 through 3 (HDDs 0-3) to the connector that is farther from the riser in which the HBA card is installed.
   2. Raise the PCIe riser green-tabbed release lever to the open (up) position and gently press the riser into the motherboard connector until it seats.
   3. Ensure that the rear bracket on the internal HBA card in PCIe slot 4 is connected to the slot in the server's chassis side wall. If the bracket is not connected, remove the riser and reposition it so that the rear bracket connects to the side wall, then gently press the riser into the motherboard connector.
   4. Slide the plastic PCIe card retainer that is mounted on the side of the chassis toward the back of the server to secure the card(s) installed in the riser.
   5. Press the green-tabbed release lever on the PCIe riser to the closed (down) position.
   6. To secure the PCIe card's rear bracket to the server, close the green-tabbed latch on the rear of the server's chassis.
7. Reconnect any external or internal cables that were previously disconnected to the replacement PCIe card.

4.Return the Server to operation

1. Replace the top cover
2. Remove any anti-static measures that were used.
3. Return the server to it's normal operating position within the rack.
4. Re-install the AC power cords and any data cables that were removed.
5. Power on server. Verify that the Power/OK indicator led lights steady on.

 5. Once the node is back up and running again, check to make sure the ifcfg-eth0 and ifcfg-eth1 files contain the correct new mac address.

Use the "ifconfig -a" command to find the new HWaddr, for example:

note *If you are running the VM environment, you will need to run these commands in DOM 0, not ORA_Base domain

# ifconfig -a
.....

eth0      Link encap:Ethernet  HWaddr A0:36:9F:1B:BE:24
          inet addr:xxx.xxx.xx.xx  Bcast:xxx.xxx.xx.255  Mask:255.255.255.0
          inet6 addr: fe80::a236:9fff:fe1b:be24/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:16716302 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15714286 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:22288336500 (20.7 GiB)  TX bytes:19017783519 (17.7 GiB)


eth1      Link encap:Ethernet  HWaddr A0:36:9F:1B:BE:26
          inet addr:xxx.xxx.xx.xx  Bcast:xxx.xxx.xx.255  Mask:255.255.255.0
          inet6 addr: fe80::a236:9fff:fe1b:be26/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:15464821 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14590017 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:19580898747 (18.2 GiB)  TX bytes:14227923533 (13.2 GiB)

If the output of ifconfig show as '__tmpxxxxxxx' instead of ethX, then refer to Doc ID 1458683.1 to resolve this issue.

The files should show the new HWaddr:

/etc/sysconfig/network-scripts/ifcfg-eth0

# Intel Corporation Ethernet Controller 10 Gigabit X540-AT2
DEVICE=eth0
HWADDR=A0:36:9F:1B:BE:24
ONBOOT=yes
HOTPLUG=no
BOOTPROTO=none
NETMASK=255.255.255.0
IPADDR=xxx.xxx.xx.xx
TYPE=Ethernet
MTU=9000

/etc/sysconfig/network-scripts/ifcfg-eth1

# Intel Corporation Ethernet Controller 10 Gigabit X540-AT2
DEVICE=eth1
HWADDR=A0:36:9F:1B:BE:26
ONBOOT=yes
HOTPLUG=no
BOOTPROTO=none
NETMASK=255.255.255.0
IPADDR=xxx.xxx.xx.xx
TYPE=Ethernet
MTU=9000

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Boot up system and verify full functionality

REFERENCE INFORMATION:
Sun Server X3-2 Documentation

http://docs.oracle.com/cd/E22368_01/index.html

References

This solution has no attachment