How to Replace a Sun Server X4-8, Oracle Server X5-8 Midplane Assembly [VCAP]

Asset ID:	1-71-1918387.1
Update Date:	2018-05-15
Keywords:

Solution Type Technical Instruction Sure

Solution 1918387.1 : How to Replace a Sun Server X4-8, Oracle Server X5-8 Midplane Assembly [VCAP]

Applies to:

Exadata X4-8 Hardware - Version All Versions to All Versions [Release All Releases]
Oracle Server X5-8 - Version All Versions to All Versions [Release All Releases]
Sun Server X4-8 - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Goal

How to Replace a Sun Server X4-8, Oracle Server X5-8 Midplane Assembly.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED?:
Sun Server X4-8, Oracle Server X5-8 Training

TIME ESTIMATE: 60 minutes

TASK COMPLEXITY: 1

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

PROBLEM OVERVIEW: A Sun Server X4-8, Oracle Server X5-8 Midplane Assembly needs replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

This is a cold-service component. Disconnect the AC power cables from the server before performing this procedure.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

Reference Doc:
Sun Server X4-8 Service Manual
http://docs.oracle.com/cd/E40591_01/html/E40317/index.html

Oracle Server X5-8 Service Manual
http://docs.oracle.com/cd/E56301_01/html/E56311/index.html

How to Remove the Midplane Assembly

1.Before replacing this component, backup the midplane FRUID information.

1.1. Log into the ILOM and check the TLI container values and sync them if needed. The server supports Top Level Identifier (TLI) and stores the TLI records in the midplane (MIDPLANE) FRUID container, called Primary, the Front Indicator Module (FIM), called Backup1, and the SMOD (SMOD/MB) board, called Backup2.

To avoid mismatched TLI values causing a failure after a Midplane replacement the fruid data should be confirmed to have matching data in at least the Primary (MIDPLANE) and both Backup1/Backup2 (FIM) and (SMOD) containers, respectively, so that the Midplane will have it's container updated automatically after replacement. Go into restricted mode and use the showpsnc command to check this.
-> set SESSION mode=restricted

WARNING: The "Restricted Shell" account is provided solely
to allow Services to perform diagnostic tasks.

[(restricted_shell) x4-8-example-sp:~]# showpsnc
Primary: fruid:///SYS/MIDPLANE
Backup 1: fruid:///SYS/FIM
Backup 2: fruid:///SYS/SMOD/MB

Element           | Primary           | Backup1           | Backup2
------------------+-------------------+-------------------+-------------------
PPN                 X4-8P1.L+NPAP1+1    X4-8P1.L+NPAP1+1    X4-8P1.L+NPAP1+1
PSN                 1421NM8004          1421NM8004          1421NM8004
Product Name        SUN SERVER X4-8     SUN SERVER X4-8     SUN SERVER X4-8
[(restricted_shell) x4-8-example-sp:~]# exit
The above example shows a system with all three containers properly in sync. If the output from the system does not show all of the containers with matching values then you should reset the SP and then re-check the values again. An ILOM reset will attempt to auto-populate the matching values if one container is out of sync.
-> reset /SP
Are you sure you want to reset /SP (y/n)? y
Performing reset on /SP
After an ILOM reset if the Primary and Backup containers match then proceed with the following steps to replace the Midplane. If these containers do not match then DO NOT proceed with the replacement yet.
If the containers do not match you will need to use the copypsnc command from service or escalation mode to copy the data from the good container so that the Primary and Backup containers match. If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to update product serial number on systems which implement TLI functionality (Doc ID 1280913.1)" and contact the TSC if needed.
How to access service mode and escalation mode on ILOM 3.x and later platforms (Doc ID 1019946.1)
After the TLI data in the Primary and Backup containers have been confirmed to match proceed with the following steps.

2. Prepare the server for cold service. Shutdown OS and power off server.

Caution - Hazardous voltage. Possibility of electrical shock if the power cables are connected to the server when accessing the midplane and bus bar assembly. This is a cold-service procedure. Disconnect all power cables before removing the server access panels.

3. Disconnect the AC power cables from the rear of the server.

4. At the front of the server, remove all eight of the fan modules.
Reference doc: How to Remove a Fan Module
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsac.html#scrolltoc

5. Remove both fan frames.
Reference doc: How to Remove fan frames
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsfo.html#scrolltoc

6. Remove the Compute Modules (CMOD).
Reference doc: How to Remove a CMOD
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsds.html#scrolltoc

7. Remove the four power supplies (PSU).
Reference doc: How to Remove a PSU
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsdt.html#scrolltoc

8. Remove the Front Indicator Module (FIM).
Reference doc: How to Remove a FIM
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsef.html#scrolltoc

9. At the rear of the server, remove any cables attached at the rear of the server. Before removing cables, label each one, noting its connection point.

10. Remove the System Management Module (SMOD).
Reference doc: How to Remove an SMOD
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsea.html#scrolltoc

11. Remove the Dual PCI Card Carriers (DPCC). Before removing the DPCCs, label each one populated with PCIe cards, noting its slot location. For slot designations, see:
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsch.html#XFESMgnsfj

Reference doc: How to Remove an DPCC
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsen.html#scrolltoc

12.Remove the eight screws that secure the server to the rack at the front of the server.

13.Use a lift to remove the server from the rack.

14.Loosen the nineteen captive screws on the top access panel and remove the panel.

15.Carefully remove and retain the protective black plastic insert that covers the top of the Midplane Assembly. Note the orientation of the insert with respect to the opening. The insert is fitted to the opening using tabs and flaps.

16.Loosen the four captive screws that secure the left side (from the front) access panel and remove the panel. The opening behind the access panel provides access to five cables (four 20-pin PSU cables and one 8-pin cable for the AC power indicator LEDs) that are routed through the server and are connected to the backside of the midplane assembly. To remove the midplane assembly, disconnect these connectors. As an aid, a label on the side of the chassis shows the arrangement and labeling of the five connectors.

[----] AC LED PWR

__ __

PS2 | | PS3

| |

__ __

PS0 | | PS1

| |

__ __

17.Reach into the side access opening and disconnect the five connectors from the back of the midplane assembly.

Tip - To see the connectors, direct the light from a flashlight through the small opening in the top access just above the cable connector points.

18.Lift the midplane assembly out of the server chassis from the top access opening.

19.Position the cables so the connectors are hanging outside of the server chassis's side access opening. The cables inside the chassis should lie flat, so they are not damaged during the installation of the replacement midplane assembly.

How to Install the Midplame Assembly

1.Align and position the replacement midplane assembly in the opening in the top of the server.

2.Carefully lower the midplane assembly into the server until the top of the assembly is flush with the top of the server. Take care not to damage the cables and connectors when lowering the midplane assembly into the server.

3.Connect the five cables to the connectors on the back of the midplane assembly. Ensure that the connectors are positioned correctly, so the key (protrusion) on the side of the connector is aligned with the notch in the connector on the midplane assembly. For best access and visibility, first attach the cable labeled PS 0, then PS 2, followed by PS 1, and finally PS 3. Attach the cable for the AC power indicator board last.

4.Position the side access cover at the access opening, ensuring that the four captive screws are aligned with the holes in the side of the chassis.

5.To secure the access cover to the chassis, tighten the four captive screws.

6.Install the black plastic insert over the top of the midplane assembly. Ensure that the orientation of the insert is correct. The insert is fitted to the opening using tabs and flaps.

7.Align the top access cover with the opening in the top of the chassis.

8.To secure the cover to the chassis, tighten the nineteen captive screws.

9.Install the chassis in the rack.

10.At the front of the server, install the CMODs.
Reference doc: How to Install a CMOD
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsec.html#scrolltoc

11.Install both fan frames.
Reference doc: How to Install a fan frame
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsff.html#scrolltoc

12.Install all eight fan modules.
Reference doc: How to Install a fan module
http://docs.oracle.com/cd/E40591_01/html/E40317/gnscm.html#scrolltoc

13.Install the FIM.
Reference doc: How to Install the FIM
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsdz.html#scrolltoc

14.Install the four PSUs.
Reference doc: How to Install the PSU
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsel.html#scrolltoc

15.At the rear of the server, install the SMOD.
Reference doc: How to Install the SMOD
http://docs.oracle.com/cd/E40591_01/html/E40317/gnser.html#scrolltoc

16.Install the DPCCs.
Reference doc: How to Install the DPCC
http://docs.oracle.com/cd/E40591_01/html/E40317/gntoz.html#scrolltoc

For slot designations, see
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsch.html#XFESMgnsfj

17.Attach cables to their respective connectors and ports at back side of the server.

18.Prepare the server for operation.

19.After replacing the Midplane Assembly in your server, you can initiate a retore of the latest TLI record information in the Backup1/Backup2 containers to the MIDPLANE FRUID on the Midplane Assembly container using this procedure.

A. Enter Service mode.
How to access service mode and escalation mode on ILOM 3.x and later platforms (Doc ID 1019946.1)

B. To restore the Primary container from the Backup1/Backup2 container, enter the following command:

[(escalation_mode) x4-8-example-sp:~]# copypsnc Backup1 Primary

C. Verify the TLI fruid data is updated on the new Midplane Assembly, as the Midplane Assembly is the Primary device. The TLI fruid data should update automatically if the Backup1 and Backup2 devices and data are valid. If it doesn't use (Doc ID 1280913.1) to update the TLI fruid data manually.

How to verify the Midplane Assembly is working properly.

1. Log into the ILOM.

2. Enter the following command to check status in normal status:

-> show System

Example:

-> show System

   Properties:
        health = OK
        health_details = -
        open_problems_count = 0
        type = Rack Mount
        model = SUN SERVER X4-8
        qpart_id = Q10548
        part_number = X4-8P1.L+NPAP1+1
        serial_number = 1421NM8004
        system_identifier = (none)
        system_fw_version = 3.2.2.12
        primary_operating_system = Not Available
        primary_operating_system_detail = Comprehensive System monitoring is not available. Ensure the host is running with the Hardware Management Pack. For details go to
                                          http://www.oracle.com/goto/ilom-redirect/hmp-osa
        host_primary_mac_address = 00:10:e0:5b:c1:8c
        ilom_address = xx.xx.xx.xx
        ilom_mac_address = 00:10:E0:5B:C1:8E
        locator_indicator = Off
        power_state = On
        actual_power_consumption = 413 watts
        action = (none)

-> show /SYS/MIDPLANE

Example:

-> show /SYS/MIDPLANE

/SYS/MIDPLANE
    Targets:

    Properties:
        type = Chassis
        ipmi_name = MIDPLANE
        product_name = SUN SERVER X4-8
        product_part_number = X4-8P1.L+NPAP1+1
        product_serial_number = 1421NM8004
        product_manufacturer = Oracle Corporation
        fru_description = ASSY,BUSBAR/MIDPLANE
        fru_manufacturer = MiTAC International Corporation
        fru_part_number = 7054513
        fru_rev_level = 07
        fru_serial_number = 489089M+14072W006N

Check ILOM event log to see if any errors exist.

faultmgmtsp> fmadm faulty

Example

-> start /SP/faultmgmt/shell
Are you sure you want to start /SP/faultmgmt/shell (y/n)? y

faultmgmtsp> fmadm faulty
No faults found

-> show /SP/logs/event/list

Check Solaris FMA info if any errors exist.

Example

fmadm faulty -a
STATE RESOURCE / UUID
-------- ----------------------------------------------------------------------

Reference Doc ID: 1381773.1 to clear the fault.

How to clear FMA logs on the ILOM or Solaris:ATR:1381773.1:1 (Doc ID 1381773.1)

Verify /var/adm/messages file has no related errors

Example

bash-3.00# cd /var/adm
# grep -i warning messages
# grep -i error messages
# grep -i fail messages

Otube video:
https://otube.oracle.com/media/How+to+Remove+and+Replace+the+Midplane+Assembly+in+an+Sun+Server+X4-8.mp4/0_gb8oxn9v

References

<NOTE:1019946.1> - How to access service mode and escalation mode on ILOM 3.x and later platforms
<NOTE:1381773.1> - How to clear FMA logs on the ILOM or Solaris on x86 platforms

Attachments

This solution has no attachment