Oracle SuperCluster T5-8 Hardware
 
SPARC T5-4
 
Oracle SuperCluster T5-8 Half Rack
 
Oracle SuperCluster T5-8 Full Rack
 
SPARC T5-8
 
Oracle Exalytics T5-8
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: SPARC-CAP VCAP
 

How to remove and replace a SPARC T5-4 or T5-8 PCI Express Card and PCIe Card Carrier

In this Document

Goal

Solution

References

Applies to:

Goal

How to Remove and Replace a SPARC T5-4 or T5-8 PCI Express Card and PCIe Card Carrier (Doc ID 1554099.1)

*********************************************************************
To report errors or request improvements on this procedure,
please Add a Comment on Doc ID: 1554099.1

*********************************************************************

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS ARE REQUIRED

No special skills required, Customer Replaceable Unit (CRU) procedure

Time Estimate: 30 minutes

Task Complexity: 0

REMOVAL/REPLACEMENT INSTRUCTIONS:

PROBLEM OVERVIEW: Replace a PCI expansion card and/or PCIe Card Carrier

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

PCIe expansion cards and PCIe Card Carrier are hot-service components that can be replaced at any time if the cards is not in use.

Note for OVM (LDOM):To remove a PCIe card that is assigned to an I/O domain, first remove the device from the I/O domain. Then, add the device to the root domain before you physically remove the device from the system. These steps enable you to avoid a configuration that is unsupported by the Direct I/O or SR-IOV feature. For more information about making hardware changes to an I/O domain, refer to the Oracle VM for SPARC documentation. Also, removing the PCI card dynamically (DR) is not supported and 'cfgadm' will not work on Physical I/O devices bound to LDOM configuration, reference the OVM Administration Guide (DR).

WHAT ACTIONS ARE REQUIRED:

Remove a PCI Expansion Card Carrier

CAUTION: Do not press on the PCI carrier filler panel when the system is powered on (OBP state included), as it can potentially cause a short circuit resulting in a "PCI Express hotplug controller detected power failure" event.

1. Take the necessary ESD precautions.
2. Locate the PCIe card carrier at the rear of the server.
3. Determine if you are removing a card carrier from a running server.

   • If you are removing a PCIe card from a server that is running (that is, if you are hot-swapping the card), go to Step 4.

   • If you are removing a card from a powered-down server, go to Step 5.

4. Determine if the PCIe card has an Attention button.

   If the PCIe card has an Attention button, you can use that button to hot-swap the card from the server. If not, you can use the CLI to hot-swap the card.

   • If the card has an Attention button, press the button to bring the card offline. The Power OK LED should go off, indicating that the card is ready to be removed. Go to Step 5.
   • If the card does not have an Attention button, take the card off line using the CLI:
     1. At the Oracle Solaris prompt, type the cfgadm -al command to list all devices in the device tree, including the PCIe cards:

        # cfgadm -al

        This command lists dynamically reconfigurable hardware resources and shows their operational status. In this case, look for the status of the card you plan to remove. This information is listed in the Occupant column.

        Example:

        Ap_id                     Type         Receptacle   Occupant        Condition
        PCIE1                     sas/hp       connected    configured      ok
        PCIE2                     sas/hp       connected    configured      ok
        ...

     2. Take the PCIe card offline using the cfgadm -c disconnect command.

        Example:

        # cfgadm -c disconnect Ap-id

        Where Ap-id is the ID of the card that you want to remove.

     3. Verify that the card's green Power LED is off.
5. Disconnect any transceivers, if applicable, and all of the cables connected to the PCIe card.

             Tip - Label the cables to ensure proper connection to the replacement card.

       6. Pull the carrier's handle down to disengage the carrier from the card cage.

         Reminder: only the green touch point at the tip of the handle should be engaged to pull down the carrier.

       7. Remove the carrier from the server.

Remove a PCIe Card from the carrier:

       1. Unlatch and open the top cover of the carrier.

       2. Carefully, remove the PCIe card from the carrier.

Install a PCIe Card

1. Unlatch and swing open the top of the PCIe card carrier.
2. Insert the PCIe card into the carrier until the bottom connector is firmly seated in the carrier's connector.
3. Close and latch the top cover on the carrier.

Install a PCIe Card Carrier

      1.  Insert the carrier into the open slot.

      2.  Close the latch to lock the carrier in place.

      3.  Reconnect all of the cables and any transceivers, if applicable, to the PCIe card.

      4. Determine if the PCIe card has an Attention button:

1. If the card has an Attention button, press the button to bring the card online.

   The card's Power OK LED should illuminate, indicating that the card is online. Go to step 5.

       If the card does not have an Attention button, type:

   # cfgadm -c connect Ap_id

       In Certain Cases in may also be required to configure the device. 

       # cfgadm -c configure AP_id

      where Ap_id is the ID of the card that you want to connect.

      5. Verify the card's installation.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTIONS ARE REQUIRED TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Verify PCIe Card Functionality:

1. Verify that the Fault LED is not lit on the PCIe card.
2. Verify that the System Service Required LEDs on the front panel and rear I/O module are not lit.
3. Verify that the System EM Fault LED on the front panel is not lit.
4. Verify that the green Power LED is lit on the card that you installed.
5. At the Oracle Solaris prompt, use the cfgadm -al command to ensure that the card is connected.

   Example:

   # cfgadm -al
   ...
   Ap_id                     Type         Receptacle   Occupant        Condition
   PCIE1                     sas/hp       connected    configured      ok
   PCIE2                     sas/hp       connected    configured      ok
   ...

If this a replacement for a faulty card, test functionality of the system:

1. Run the Solaris "fmadm faulty" and SP/ILOM "show faulty" command (if only ALOM is supported run "showfaults -v" command) to verify that the fault has been cleared.
2. Perform one of the following tasks based on your verification results:
* If the previous steps did not clear the fault, refer to doc 1004229.1 for information about the tools and methods you can use to diagnose and clear component faults.
* If the previous steps indicate that no faults have been detected, the component has been replaced successfully. No further action is required
3. Restart software applications per applicable administration guides to resume system operation.

PARTS NOTE:
for T5-4 refer to: https://support.oracle.com/handbook_private/Systems/SPARC_T5_4/components.html

for T5-8 refer to: https://support.oracle.com/handbook_private/Systems/SPARC_T5_8/components.html

REFERENCE INFORMATION:
SPARC T5-4 Service Manual: http://docs.oracle.com/cd/E29659_01/pdf/E29663.pdf

SPARC T5-8 Service Manual: https://docs.oracle.com/cd/E35078_01/html/E35082/index.html

See also:  Oracle Integrated Lights Out Manager (ILOM) 3.2 Documentation Collection: http://docs.oracle.com/cd/E37444_01/index.html

References

This solution has no attachment