Mx-32 - How to Replace a Faulty PCIe Hot-Plug Carrier (CAR)

Asset ID:	1-71-1533492.1
Update Date:	2016-03-16
Keywords:

Solution Type Technical Instruction Sure

Solution 1533492.1 : Mx-32 - How to Replace a Faulty PCIe Hot-Plug Carrier (CAR)

Applies to:

SPARC M5-32 - Version All Versions and later
SPARC M6-32 - Version All Versions and later
Oracle SuperCluster M6-32 Hardware - Version All Versions and later
Information in this document applies to any platform.

Goal

CAP PROBLEM OVERVIEW: Mx-32 - How to Replace a Faulty PCIe Hot-Plug Carrier (CAR)

To report errors or request improvements on this procedure, please go to My Oracle Support, and put a comment on Doc ID: 1533492.1

ESD Caution:

Circuit boards and drives contain electronic components that are extremely sensitive to static electricity. Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these boards. Do not touch the components along their connector edges.
Use a Antistatic Wrist strap. Attach one end of the strap to your wrist and the other end to the chassis, depending on what type of strap you use, with the adhesive end or the metal plug.
Use an Antistatic Mat. Place ESD-sensitive components such as motherboards, memory, and other PCBs on an antistatic mat.

Contamination Caution:

Dust particles of packaging material are number one cause of datacenter contamination. Make sure to remove all packaging material, up to the ESD safe packaging material, while still being outside the datacenter.

HOT Replacement Caution:

Oracle SuperCluster M6-32 Hardware does not support PCIe card hot replacement (aka hotplug). The domain which owns the PCIe slot must be shutdown to replace the PCIe carrier.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE ENGINEER NEED:(IS A SITE ENGINEER AVAILABLE?)

M5-32 Product Training/Experience

TASK COMPLEXITY: 1

TIME ESTIMATE: 30 minutes

HOT replacement

FIELD ENGINEER INSTRUCTIONS

Prepare labeling materials sufficient for all IO cables attached to the IOU.

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? : Solaris must be running in the domain which owns the target PCIe slot in order to use HOT replacement and the ATTN button

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

Note IOU Slots are numbered left to right and the PCIe slot numbers are labeled from bottom to top.

1. Press the ATTN button on the carrier that contains the I/O card that you wish to remove.
    The LEDs on the carrier flash for approximately 10 seconds as the PDomain disables the I/O card. When the LEDs on both the carrier and the card turn off, the carrier and card are ready to remove.
    If the Green OK LED fails to extinguish the I/O card cannot safely be removed. Investigation must be made via Solaris to determine why the card has not been offlined. Domain shutdown may be required to
    safely removed the I/O card.

NOTE: The ATTN button will only function if Solaris is running in the domain which owns the PCIe slot.

2. Label and remove any I/O cables from the I/O card.
3. Remove the carrier from the slot:
   a. Pull the carrier’s extraction lever. ( The lever is held in place by friction. )
   b. Swing the extraction lever out 90 degrees until the far end of the lever begins to push the carrier out of the slot.
   c. Remove the carrier from the slot.
   d. Place the carrier on a static-safe workspace.

Remove the I/O card from the carrier.
   a. Lift the green tab to unlock and open the top of the PCIe hot-plug carrier
   b. Pull the I/O card or filler panel out of the carrier.

Install the I/O Card in the new PCIe Hot-Plug Carrier

1. Seat the I/O card or filler panel in the PCIe hot-plug carrier.
2. Close the top of the carrier.
   The green latch should click into place. If the top is difficult to close, verify that the notch of the card bracket fits around the guide post.

Install the carrier in the server.

1. Insert the PCIe hot-plug carrier with the I/O card in the slot and lock the carrier’s extraction lever

Push evenly on both sides of the carrier so that the carrier slides straight into the slot
If the carrier slides correctly into the slot, you should feel a slight resistance as the carrier starts to seat in the connector
Do not push the extraction lever while you insert the carrier into the slot. The carrier can enter at an angle and damage the connectors
Lock the carrier's extraction lever
The LEDs on the carrier and the card should remain off at this point

2. Attach I/O cables to the card.

3. Press the ATTN button on the carrier to reconfigure the I/O card into the PDomain.
The carrier’s LEDs should flash for a few seconds until PDomain enables the I/O card. The card’s LEDs will show activity when the card is enabled

Return the faulted component to Oracle.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Restart software applications per applicable administration guides to resume system operation.

If the PCIE slot fails to be available once Solaris is booted the following issue may apply. Consult
FMA I/O retirement : PCI devices can be seen from OBP but disappear when System Boots up into Solaris (Doc ID 1614738.1)
NOTE: make certain to check for the /etc/devices/retire_store within the control domain which consumes subscribed IO fault diagnosis.

======================== Other info =====================

REFERENCE INFORMATION: Service Manual: https://www.oracle.com/technetwork/documentation/oracle-sparc-ent-servers-189996.html

Attachments

This solution has no attachment