How to Replace a Sun Server X4-8, Oracle Server X5-8 Dual PCIe Card Carrier (DPCC) and/or PCIe Card

Asset ID:	1-71-2007023.1
Update Date:	2018-05-15
Keywords:

Solution Type Technical Instruction Sure

Solution 2007023.1 : How to Replace a Sun Server X4-8, Oracle Server X5-8 Dual PCIe Card Carrier (DPCC) and/or PCIe Card

Applies to:

Sun Server X4-8 - Version All Versions to All Versions [Release All Releases]
Oracle Server X5-8 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Goal

How to Replace a Sun Server X4-8, Oracle Server X5-8 Dual PCIe Card Carrier (DPCC) and/or PCIe Card.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED:
No special skills required, Customer Replaceable Unit (CRU) procedure

TIME ESTIMATE: 30 minutes

TASK COMPLEXITY: 0

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

PROBLEM OVERVIEW: A Sun Server X4-8, Oracle Server X5-8 Dual PCIe Card Carrier (DPCC) and/or PCIe Card needs replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

System should be powered down.

WHAT ACTIONS ARE REQUIRED?:

Reference Doc:

Remove a DPCC:
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsen.html#scrolltoc

Note: The DPCCs can be removed with the server OS running, but the hot-pluggability of the PCIe cards is OS and driver dependent which is outside the scope of this replacement action plan.

How to Remove a DPCC

1. Prepare the server for service, shutdown OS and power off server.

2. Identify the DPCC to be removed.

3. To unlock the DPCC lever, lift the release latch and pull the lever downward, away from the server. This action disengages the PCIe card IO connectors from the connectors on the back of the CMODs.

4. To remove the DPCC, slide it out of the server.

Remove a PCIe Card

1. Orient the DPCC so that the hinge is to the left.

2. To open the top of the DPCC, lift the release latch at the non-hinged end of the lid and rotate the lid upward and to the left.

3. To remove the card, pull it straight up and out of its connector.

4. Repeat steps 3 for the second PCIe card in the DPCC if replacing the DPCC.

Install a PCIe Card

1. Identify the DPCC PCIe slot.

2. Orient the DPCC so that the hinge is to the left.

3. Ensure that the DPCC top cover is open. The top of the DPCC is hinged at one end. To open, lift the tab on the non-hinged end.

4. Orient the PCIe card with the edge (or bus) connector facing downward and the IO (or cable connector) facing to the left. This puts the component side of the card facing away from you.

5. To install the card, align the edge connector with the slot in the DPCC and push the card downward into the slot.

6. To close the top of the DPCC, rotate it to the right ensuring the clip on the edge of the top is secured over the unhinged edge of the DPCC.

Caution - Pinch point. Keep fingers away from the underside of the top when closing it.

Install the DPCC

1. Ensure the top of the DPCC is closed and secured and the lever on the front of the DPCC is in its fully open position.

2. Align the DPCC with the vacant slot. The connector (back) side of the DPCC faces inward toward the server.

3. Slide the DPCC into the slot until it stops. This leaves the DPCC protruding slightly from the back of the server. Do not attempt to push the DPCC inward beyond this point.

4. Rotate the lever on the DPCC upward until it locks into place. This action draws the DPCC inward engaging the connectors in the DPCC with the connectors on the server midplane.

Caution - Pinch point. Keep fingers away from the backside of the lever when closing it

5. Prepare server for operation, power on server and start OS.

6. Verify that the green OK indicators on the front of the DPCC are on steady.

WHAT ACTIONS ARE REQUIRED TO RETURN THE SYSTEM TO AN OPERATIONAL STATE?:

How to verify the DPCC is working properly

Log in to the ILOM CLI.

Enter the following command to check status is normal for DPCC:

-> show /SYS/DPCC0

/SYS/DPCC0
    Targets:
        PCIE1
        PCIE2
        PRSNT

    Properties:
        type = PCIE Module
        ipmi_name = DPCC0

    Commands:
        cd
        show

->

To check the status of a PCI card use these commands:

-> show /SYS/DPCC0/PCIE1

/SYS/DPCC0/PCIE1
    Targets:
        PRSNT

    Properties:
        type = PCIE Module
        ipmi_name = PCIE1
        fault_state = OK
        clear_fault_action = (none)

    Commands:
        cd
        set
        show

->

-> show /System/PCI_Devices/Add-on/Device_1

/System/PCI_Devices/Add-on/Device_1
    Targets:

    Properties:
        part_number = 7107091
        description = Sun Flash Accelerator F80 800GB eMLC PCI-E 2.0 Low
                      Profile Adapter
        location = PCIE1 (PCIe Slot 1)
        pci_vendor_id = 0x1000
        pci_device_id = 0x007e
        pci_subvendor_id = 0x108e
        pci_subdevice_id = 0x050a

    Commands:
        cd
        show

->

Use the fault management shell to clear the fault for a specific component:

For example:

faultmgmtsp> fmadm acquit /SYS/DPCC0/PCIE1

or you can use the UUID

faultmgmtsp> fmadm acquit UUID

Check if any error output from event log

Example

-> show /SP/logs/event/list

Check if any faulted parts exist.

-> start /SP/faultmgmt/shell
Are you sure you want to start /SP/faultmgmt/shell (y/n)? y

Example

faultmgmtsp> fmadm faulty
No faults found

Check FMA and OS information to verify that no errors existed

Example

# fmadm faulty -a
STATE RESOURCE / UUID
-------- ----------------------------------------------------------------------

# prtdiag -v
System Configuration: Sun Server X4-8
BIOS Configuration: American Megatrends Inc. 29011300 08/26/2010
BMC Configuration: IPMI 2.0 (KCS: Keyboard Controller Style)

==== Processor Sockets ====================================

Version Location Tag
-------------------------------- --------------------------
Intel(R) Xeon(R) CPU E7-8895 v2 @ 2.80GHz CPU 1
Intel(R) Xeon(R) CPU E7-8895 v2 @ 2.80GHz CPU 2
Intel(R) Xeon(R) CPU E7-8895 v2 @ 2.80GHz CPU 3
Intel(R) Xeon(R) CPU E7-8895 v2 @ 2.80GHz CPU 4
.
==== Upgradeable Slots ====================================

ID Status    Type             Description
--- --------- ---------------- ----------------------------
1   in use    PCI Express Gen3 x16 /SYS/DPCC0/PCIe1
2   in use    PCI Express Gen3 x86 /SYS/DPCC0/PCIe2
3   available PCI Express Gen3 x16 /SYS/DPCC1/PCIe3
4   in use    PCI Express Gen3 x86 /SYS/DPCC1/PCIe4
5   available PCI Express Gen3 x16 /SYS/DPCC2/PCIe5
6   in use    PCI Express Gen3 x86 /SYS/DPCC2/PCIe6
7   available PCI Express Gen3 x16 /SYS/DPCC3/PCIe7
8   in use    PCI Express Gen3 x86 /SYS/DPCC3/PCIe8
.
.

Check /var/adm/messages file to verify that no errors existed

Example

bash-3.00# pwd
/var/adm

# grep -i warning messages
# grep -i error messages
# grep -i fail messages

PARTS NOTE:

REFERENCE INFORMATION:
Sun Server X4-8 Service Manual
https://docs.oracle.com/cd/E40591_01/html/E40317/index.html

Oracle Server X5-8 Service Manual
https://docs.oracle.com/cd/E56301_01/html/E56311/index.html

How to Shutdown and Startup Exadata X5 compute nodes and storage cells when performing hardware maintenance (includes Supercluster X5 storage cells) (Doc ID 1982342.1)

References

<NOTE:1381773.1> - How to clear FMA logs on the ILOM or Solaris on x86 platforms

Attachments

This solution has no attachment