Sun Server X4-8
 
Oracle Server X5-8
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

In this Document

Goal

Solution

References

Applies to:

Goal

How to Replace a Sun Server X4-8, Oracle Server X5-8 System Module (SMOD).

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED?:
Sun Server X4-8, Oracle Server X5-8 Training

TIME ESTIMATE: 45 minutes

TASK COMPLEXITY: 1

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

PROBLEM OVERVIEW: A Sun Server X4-8, Oracle Server X5-8 System Module (SMOD) needs replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

This is a cold-service component. Disconnect the AC power cables from the server before performing this procedure.

WHAT ACTIONS ARE REQUIRED?:

Reference Doc:
Sun Server X4-8 Service Manual
http://docs.oracle.com/cd/E40591_01/html/E40317/index.html

Oracle Server X5-8 Service Manual
http://docs.oracle.com/cd/E56301_01/html/E56311/index.html

Sun Server X4-8 System Module (SMOD) Overview
http://docs.oracle.com/cd/E40591_01/html/E40317/gnsfb.html#XFESMgnzll

The SMOD motherboard hosts the service processor (SP), two disk backplanes (for the externally accessible server storage drives), and the system battery. It also has a PCIe riser for the server storage HBA and the two internal USB ports. The PCIe riser is located on the bottom of the SMOD.

FPGA firmware resides on both the SP and the CPU modules (CMOD). The FPGA firmware versions on both components must match. You can perform firmware updates only on the SMOD.  The FPGA firmware that resides on the CMODs is maintained by the SMOD. The SMOD automatically updates all CMODs to its version of FPGA firmware. If a CMOD is replaced and its version is different from the FPGA firmware version that resides on the SMOD, then the SMOD will update or downgrade the new CMOD to the version of FPGA firmware that resides on the SMOD. If the SMOD module is replaced and its FPGA firmware is different from the firmware on the CMODs, it will update or downgrade the CMODs to the version of FPGA firmware that resides on the SMOD.

The following table shows the resulting SMOD response to user-initiated firmware updates and component replacements.

User Action                                        SP Response

How to Remove the SMOD Module

1. Since the SMOD contains the Service Processor (SP), before shutting down the server make sure to backup the ILOM configuration, and make sure to note the current ILOM + BIOS levels.  You will need to update the replacement SMOD if not at this level.

1. As long as the system's ILOM is functional and not the reason for the SMOD replacement the ILOM configuration should be backed up so that it may be restored to the new replacement SMOD. If the ILOM is not accessible and cannot be backed up the FE will need to re-configure the ILOM manually after the SMOD replacement.
2. To backup the settings using the ILOM web browser expand the "ILOM Administration" section in the left column and then select "Configuration Management". In the main panel you may then backup the ILOM settings.
3. To backup the settings using the ILOM cli you must use scp to copy the file to an available system on the admin network (or your laptop if you connect it directly to the ILOM net management port and configure it for the same network). Use the following commands substituting the appropriate values for the login name, password, ipaddress and file destination. The passphrase should be set to the ILOM's password.
   -> cd /SP/config
   -> set passphrase=ilompassswd
   -> set dump_uri=scp://username:password@IPaddress/tmp/SP.config

1.1. While still logged into the ILOM check the TLI container values and sync them if needed.  The server supports Top Level Identifier (TLI) and stores the TLI records in the midplane (MIDPLANE) FRUID container, called Primary, the Front Indicator Module (FIM), called Backup1, and the SMOD (SMOD/MB) board, called Backup2.

1. To avoid mismatched TLI values causing a failure after a SMOD replacement the fruid data should be confirmed to have matching data in at least the Primary (MIDPLANE) and the Backup1 (FIM) containers so that the SMOD will have it's container updated automatically after replacement. Go into restricted mode and use the showpsnc command to check this.  
   -> set SESSION mode=restricted
   
   WARNING: The "Restricted Shell" account is provided solely
   to allow Services to perform diagnostic tasks.
   
   [(restricted_shell) x4-8-example-sp:~]# showpsnc
   Primary: fruid:///SYS/MIDPLANE
   Backup 1: fruid:///SYS/FIM
   Backup 2: fruid:///SYS/SMOD/MB
   
   Element           | Primary           | Backup1           | Backup2
   ------------------+-------------------+-------------------+-------------------
   PPN                 X4-8P1.L+NPAP1+1    X4-8P1.L+NPAP1+1    X4-8P1.L+NPAP1+1
   PSN                 1421NM8004          1421NM8004          1421NM8004
   Product Name        SUN SERVER X4-8     SUN SERVER X4-8     SUN SERVER X4-8 
   [(restricted_shell) x4-8-example-sp:~]# exit
    
2. The above example shows a system with all three containers properly in sync. If the output from the system does not show all of the containers with matching values then you should reset the SP and then re-check the values again. An ILOM reset will attempt to auto-populate the matching values if one container is out of sync.  
   -> reset /SP
   Are you sure you want to reset /SP (y/n)? y
   Performing reset on /SP
    
3. After an ILOM reset if the Primary and Backup1 containers match then proceed with the following steps to replace the SMOD. If these two containers do not match then DO NOT proceed with the replacement yet.
4. If the containers do not match you will need to use the copypsnc command from service or escalation mode to copy the data from the good container so that the Primary and Backup1 containers match (Backup2 is the SMOD and we are about to replace this so it is not as important at this step). If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to update product serial number on systems which implement TLI functionality (Doc ID 1280913.1)" and contact the TSC if needed.
5. How to access service mode and escalation mode on ILOM 3.x and later platforms (Doc ID 1019946.1)
6. After the TLI data in the Primary and Backup1 containers have been confirmed to match proceed with the following steps.

2. Prepare the server for cold service.

3. Access the rear of the server.

4. Label and disconnect all cables from the SMOD.

5. Disengage the SMOD from the server midplane.

    A. To Unlock the SMOD handles, squeeze together the release latches on the end of both handles.  The unlocking action is accompanied by an audible click.

    B. To disengage the SMOD from the server midplane, simultaneously, rotate both handles downward toward their fully open positions.  This action disengages the connectors on the SMOD from the connectors on the server midplane.

6. To remove the SMOD, grab it by it's sides and slide it out of the server.


How to Install the SMOD Module

1. Make sure that all removable components have been transferred to the new SMOD from the old SMOD (SAS HBA, BBU, disks, USB thumb drives, and SAS cables).

2. Ensure that the handles on the SMOD are in their fully open position.

    A. To Unlock the SMOD handles, squeeze together the release latches on the end of both handles.  The unlocking action is accompanied by an audible click.

    B. To open, rotate both handles downward until they are at a 90 degree angle to the SMOD (fully open position).

3. Orient the SMOD with the handles facing away from the server and the connectors facing toward the open slot in the server.

4. Align the SMOD in the slot.

5. Slide the SMOD into the slot until it stops. This leaves the SMOD protruding slightly from the back of the server. Do not attempt to push the SMOD inward beyond this point.

6. To install the SMOD, simultaneously rotate both handles upward until they lock into place. This action draws the SMOD inward engaging the SMOD connectors with the connectors on the server midplane.

7. Connect the external cables to the SMOD.

8. Configure the new SMOD for operation.

1. Once the AC cables are connected the SP will begin to initialize POST.  Connect to the SER MGT port and login with default root/changeme once the SP is fully booted.
2. Configure an IP address so a network connection can be made to the NET MGT port.
3. Check the ILOM + BIOS level on the new SMOD.  If a different version from the previous version upgrade/downgrade the firmware to match the previous version.  Note, the firmware must be at the same level as the previous SMOD version or the saved configuration backup may not restore all settings, or may not work at all.
4. Use the previously saved ILOM backup, and restore the ILOM configuration.
   1. To restore the settings using the ILOM web browser click ILOM Administration > Configuration Management > Backup/Restore.  Click Restore in the Operations box.  Click the Transfer Method box to specify a method for transferring the Oracle ILOM configuration file.  If the backup configuration file was encrypted with a passphrase, type the passphrase in the Passphrase text box, and then retype the passphrase in the Confirm Passphrase text box.  Click Run to initiate the Restore operation.
   2. To restore the settings using the ILOM cli navigate to the config CLI target "/SP/config".  If the backup configuration file was encrypted with a passphrase, set the value for the passphrase property to the passphrase used to encrypt the file.  To initiate the Restore operation set the load_uri, see example below:
      -> cd /SP/config
      -> set passphrase=ilompassswd
      -> set load_uri=scp://username:password@IPaddress/tmp/SP.config
5. Log back in to the ILOM and make sure the ILOM settings have been restored.

9. After adding or replacing a component in your server, you can initiate a backup of the latest TLI record information in the MIDPLANE FRUID Primary container to Backup2/Backup1 on the SMOD or FIM containers using this procedure.

    A. Enter Service mode.
How to access service mode and escalation mode on ILOM 3.x and later platforms (Doc ID 1019946.1)

    B. To backup the Primary container to the Backup2 container, enter the following command:

    C. Verify the TLI fruid data is updated on the new SMOD MB, as the SMOD is the Backup2 device.  The TLI fruid data should update automatically if the Primary and Backup1 devices and data are valid.  If it doesn't use (Doc ID 1280913.1) to update the TLI fruid data manually. 

WHAT ACTIONS ARE REQUIRED TO RETURN THE SYSTEM TO AN OPERATIONAL STATE?:

Boot up system and verify full functionality

How to verify the SMOD and Service Processor work properly.

Connect and try to access the SP via SER MGT port or after some time the SP IP address, if DHCP is available.

If the SP login is accessible, then check system status.

-> show System

Example:

Check the ILOM version:

-> show /SP

Example

Check ILOM event log to see if any errors exist.

faultmgmtsp> fmadm faulty

Example

-> show /SP/logs/event/list


Check Solaris FMA info if any errors exist.

Example

References

This solution has no attachment