MiniCluster S7-2 Hardware
 
SPARC S7-2
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: SPARC-CAP VCAP
 
Microlearning>Video>ML-VID-VCAP
 

In this Document

Goal

Solution

References

Oracle Confidential PARTNER - Available to partners (SUN).
Reason: Sonoma-T CAP

Applies to:

Goal

How to Replace a SPARC S7-2 (1 or 2 Processor) Motherboard and SP (Embedded Service Processor)

***************************************************************************************************************************
To report errors or request improvements on this procedure,please go to http://mosemp.us.oracle.com and put a comment on Doc ID: 2134593.1
***************************************************************************************************************************

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER NEED:
SPARC S7-2 Product Training required, to be serviced by qualified Oracle Service personnel, requires the ability to follow steps similar to what is in the Product Service manual.

Time Estimate: 120 minutes

TASK COMPLEXITY: 1

FIELD ENGINEER INSTRUCTIONS   

 
PROBLEM OVERVIEW: SPARC S7-2 (1 or 2 Processor) Motherboard and SP (Embedded Service Processor) Replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY?:

Customer should perform an orderly and graceful shutdown of applications and OS to get the OpenBoot PROM prompt. Run the printenv command and make a note of any OpenBoot PROM variables that have been modified. Then power off the server and remove the AC power cords from the system.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

Verify/Update TLI Prior to Replacement

1. Log into the ILOM and check the fruid container values and sync them if needed.

a. To avoid mismatched fruid values causing a failure after a motherboard replacement the fruid data should be confirmed to have matching data in at least the Primary (DBP) and Backup1 (PS0) containers so that the motherboard will have its container updated automatically after replacement. Go into restricted mode and use the showpsnc command to check this.

b. The above example shows a system with all three containers properly in sync. If the output from the system does not show all of the containers with matching values then you should reset the SP and then re-check the values again. An ILOM reset will attempt to auto-populate the matching values if one container is out of sync.

c. After an ILOM reset if the Primary and Backup1 containers match then proceed with the following steps to replace the motherboard. If these two containers do not match then DO NOT proceed with the replacement yet.
d. If the containers do not match you will need to use the copypsnc command from service or escalation mode to copy the data from the good container so that the Primary and Backup1 containers match (Backup2 is the motherboard and we are about to replace this so it is not as important at this step).

If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to update product serial number on systems which implement TLI functionality (Doc ID 1280913.1)" and contact the TSC if needed. How to access service mode and escalation mode on ILOM 3.x and later platforms (Doc ID 1019946.1).
e. After the fruid data in the Primary and Backup1 containers have been confirmed to match proceed with the following steps.

Remove Motherboard (w/ Embedded Service Processor) 

1. Prepare the server for service.
a. Power off the server and disconnect the power cords from the power supplies.
b. Extend the server to the maintenance position in the rack.
c. Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis.

2. Remove the top cover and open the fan door to remove all of the fan modules.
a. Open the fan Door. Slide the fan door latches forward and swing the door up to the open position
b. Using your thumb and forefinger, grasp each of the fan modules in the finger recesses located in the plastic between the fans and lift the fan modules straight up and out of the chassis.

c. To open the server top cover, press and hold down the top cover release button and use the recessed area to slide the top cover toward the rear of the server about 0.5 inches (12.7 mm).
d. Lift the cover off the chassis and set it aside.

3. Remove clear plastic air duct assembly cover:
a. Insert fingers in access holes at either end and pull straight up and out.

4. Remove all PCIe cards.

 A. Removing PCIe cards in slots 1 and 2:
    1. If a PCIe card is installed in the riser, disconnect any external cables from the card.
    2. Lift the green-tabbed latch on the rear of the server chassis next to the applicable PCIe slot to release the rear bracket on the PCIe card

    3. Lift the green-tabbed riser release lever on the PCIe riser with one hand and use your other hand to remove the riser from the motherboard connector
    4. If there is a PCIe card installed in the riser, place the riser on an antistatic mat and note the slot in which the PCIe riser was originally installed; otherwise, set the PCIe riser aside.

 B. Removing PCIe cards from slots 3 and 4:

     1. If there is a PCIe card installed in the riser, disconnect any external or internal cables.

     2. Open the green-tabbed latch located on the rear of the server chassis next to PCIe slot 3 to release the rear bracket on the PCIe card

      a. To release the riser from the motherboard connector, lift the green-tabbed release lever on the PCIe riser to the open position
      b. Slide the plastic PCIe card retainer, which is mounted on the side of the chassis, toward the front of the server to release the card(s) installed in the riser
      c. Grasp the riser with both hands and remove it from the server
      d. Disconnect the SAS storage drive (HDD) cables from the internal HBA card installed in PCIe slot 4. This can be done by releasing the connector catch from underside of the card.
      e. Place the riser on an antistatic mat.

5. If you are replacing the motherboard, remove the following components and place them on an ESD mat:
  a. Remove all DIMMs on motherboard.
  b. Install all DIMMs on new motherboard

6. Remove the server mid-wall
  a. Using a screwdriver (No. 2 Phillips or flathead), loosen the four green captive screws that secure the mid-wall to the server chassis.
  b. Lift up the mid-wall slightly to disengage it from the raised mushroom-shaped standoffs that are located on the server chassis sidewall (one on each end of the mid-wall), then lift it out of  the  server and set it aside.

7. Remove the HBA SAS Cable Assembly
  a. Step 5 f had the SAS cables removed from the HBA end
  b. Carefully pull SAS cables up and out of the way of mother board replacement leaving disk backplane end still connected

8. Remove the NVMe Cables:
  a. Disconnect the cables from the motherboard
  b. Carefully pull the NVMe card cables from the side of the chassis and lay them away from the mother board

9. Removing Power Supplies

  a. Disconnect the power cord from the faulty power supply.
  b. Grasp the power supply handle and push the power supply latch to the left
  c. Pull the power supply out of the chassis

10. Internal cables
  a. To disconnect the disk backplane power cable from the motherboard, press in on the connector latch and pull the connector out
  b. To eject the disk backplane data ribbon cable and front indicator module ribbon cable connectors, open both side latches

11. Remove the motherboard from the server chassis
  a. Loosen the one green captive screw that secures the motherboard to the chassis
  b. Grasp the metal bracket located just to the rear of the DIMM sockets and slide the motherboard toward the front of the server and lift it slightly to disengage it from the standoffs that are located on the server chassis under the motherboard
  c. Lift the motherboard out of the server chassis and place it on an antistatic mat and next to the replacement motherboard.

12. Remove the eUSB drive from the motherboard and install it on the replacement motherboard.
  a. Loosen the screw that attaches the eUSB drive to the motherboard.
  b. Lift the eUSB drive up to disconnect it from the motherboard.

13. Remove the SEEPROM from the motherboard and install it on the replacement motherboard

Install Motherboard (w/ Embedded Service Processor)

1. Insert the motherboard into the server chassis
 a. Grasp the metal bracket located to the rear of the DIMMs and tilt the front of the motherboard up slightly and push it into the opening in the rear of the server chassis
 b. Lower the motherboard into the server chassis and slide it to the rear until it engages the seven mushroom-shaped standoffs located on the server chassis under the motherboard.
 c. Ensure that the indicators, controls, and connectors on the rear of the motherboard fit correctly into the rear of the server chassis.
 d. Fasten the one green captive screw that secures the motherboard to the chassis

2. Internal cables
 a. To connect the disk backplane power cable to the motherboard, press in on the connector latch and push the connector in
 b. Insert the disk backplane data ribbon cable  and front indicator module ribbon cables into their appropriate  connectors

3. Installing Power Supplies

 a. Push each power supply into the chassis fully until you feel it click into position
 b. Do not connect AC power cords at this time

4. Installing NVMe Cables
  a. Connect the two NVMe cables to the mother board. The longer of the two cables should go to the 0+1 connector of the mother board

5. Installing PCIe Cards
  A. Installing PCIe card in slots 1 and 2
  1. Determine into which slot you will install the PCIe riser so it will be in the same slot from which it was removed
  2. Raise the green-tabbed release lever on the PCIe riser to the open (up) position
  3. Gently press the riser into the motherboard connector until it is seated, and then press the green-tabbed release lever on the PCIe riser to the closed (down) position
  4. Close the green-tabbed latch on the rear of the server chassis next to the applicable PCIe slot to secure the rear bracket on the PCIe card to the server chassis

  5. If there were any external cables connected to the PCIe card, reconnect them
  B. Installing PCIe card in slots 3 and 4
  1. Reconnect the SAS cables to the internal host bus adapter card from the underside of card making sure they are latched in securely
  2. Raise the green-tabbed release lever on the PCIe riser to the open (up) position, and then gently press the riser into the motherboard connector until it is seated
  3. Ensure that the rear bracket on the internal HBA card in PCIe slot 4 is connected to the slot in the server chassis side wall.
  4. If the bracket is not connected, remove the riser and reposition it so that the rear bracket connects to the side wall, and then gently press the riser into the motherboard connector.
  5. Slide the plastic PCIe card retainer, which is mounted on the side of the chassis, toward the back of the server to secure the card(s) installed in the riser
  6. Press the green-tabbed release lever on the PCIe riser to the closed (down) position
  7. To secure the rear bracket on the PCIe card to the server, close the green-tabbed latch on the rear of the server chassis

  8. If there is a PCIe card installed in slot 3 of the riser, reconnect any external or internal cables to the card

6. Installing the server mid-wall
a. Using a screwdriver (No. 2 Phillips or flathead), tighten the four green captive screws that secure the mid-wall to the server chassis. Be careful to route cables through openings without pinching them between mid-wall and chassis

7. Install Fans
a. Press down on the fan module and apply firm pressure to fully seat the fan module.

8. Take the clear plastic air duct assembly and place it over the top of the CPU's and DIMMs allowing the DIMMs to protrude through the open slots and the assembly to fit nicely between the chassis walls, power supply metal casing and mid-wall.

9. Install the server top cover and close the fan door:
a. Close the fan door by swinging the door down to the closed position and hearing the latches latch.
b. Place the server top cover on the chassis and slide the top cover toward the front of the server (about 0.5 inches (12.7 mm)) until you hear the top cover release button lock into place.

10. Return the Server to operation.
a. Remove any anti-static measures that were used.
b. Return the server to its normal operating position within the rack.
c. Re-install the AC power cords and any data cables that were removed.

11. Prior to powering on the server, connect a terminal or a terminal emulator (PC or workstation) to the  SER MGT port.
a. If needed, configure the SP network port to enable the firmware image to be downloaded. Refer to the Oracle ILOM documentation for network configuration instructions.
b. Download the system firmware. Follow the firmware download instructions in the Oracle ILOM documentation

12. Power on server. Verify that the Power/OK indicator led lights steady on.
13. When you replace a motherboard (since it the contains the SP) you will need to download the fallback miniroot image to match your Customers OS version (you can get your Customers OS version from an uploaded explorer) from My Oracle Support at https://support.oracle.com.
Then follow the procedure "Uploading a New Solaris Miniroot Package From SP to Host" in the Oracle ILOM Administrator's Guide for Configuration and Maintenance

Verify/Update TLI After Replacement

1. Set the system serial number/fruid data if needed.
a. The motherboard is not the primary fruid container in this server so when it is replaced you should not normally need to fix the serial number information (TLI).
b. login to the ILOM as root and then enter the "restricted shell" to check the fruid values (Follow Step 1a of "Verify/Update TLI Prior to Replacement" section at the beginning of CAP).
c. When the motherboard is replaced the Backup2 fruid container will likely not match the Primary entry. If it does not you must enter escalation or service mode to fix it (if all three entries match this step is done).
d. Contact the TSC to request an escalation password (service mode will work also if just the copypsnc command ends up needing to be used, if the setpsnc command is needed escalation mode is required. setpsnc is not covered in this procedure).
e. Provide your TSC contact the output from the following ILOM commands- "version", "show /SYS product_serial_number", and "show /SP/clock". If the product_serial_number information does not give good output then provide the showpsnc output that was seen in step b above as well.
f. The TSC will provide an escalation password that is made up of 32 short words. Follow the example below to create a new user with the 'Service' role assigned. The Service role is required to access service or escalation modes. In the following example we will create a user named 'escuser' with the service role.

 g. Set the check_physical_presence to false and then exit from the ILOM so that you can login as the newly created user.

h. Login using the escuser login and enter escalation mode using the password that was provided by the TSC.

i. Use the showpsnc command to confirm the current container values. Confirm that the primary container has a serial number (the value on the PSN line) that matches the system serial number. The system serial number can be checked by comparing to the serial number RFID tag on the front left hand side of the server. After confirming that there is a valid fruid primary use the copypsnc command to write the good data from the primary to the backup2 container on the MB. The following example shows copying from primary to the backup2, but you could also copy from backup1 if needed.

j. At this point if all of the fruid containers match and have the correct serial number data this step is done. If more than one of the fruid containers had non-valid entries then the copypsnc command should be used to copy over the valid data to the other container that is not valid. (ie. "copypsnc Primary Backup1") After confirming all fruid data is correct reset the ILOM to confirm that the fruid data persists through a reboot and remove the escalation user if needed.

k. If trouble is encountered during any of the steps of accessing escalation mode and fixing the fruid containers please contact the TSC for assistance.

How to verify the Motherboard is working properly

1. Log into ILOM to confirm if motherboard status is working properly.

Sample:

 2.  Check ILOM event log to see if any error related motherboard.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE FE/CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:
Boot system and monitor boot sequence for errors. Test functionality of system:
1. Run the Solaris "fmadm faulty" and SP/ILOM "show faulty" command to verify that the fault has been cleared.
2. Perform one of the following tasks based on your verification results:
  -If the previous steps did not clear the fault, refer to doc 1004229.1 for information about the tools and methods you can use to diagnose and clear component faults.
  -If the previous steps indicate that no faults have been detected, the component has been replaced successfully. No further action is required
3. Restart software applications per applicable administration guides to resume system operation.

PARTS NOTE:
https://support.oracle.com/handbook_partner/Systems/SPARC_S7_2/components.html#SystemBoard

REFERENCE INFORMATION:
SPARC S7-2 Service Manual: http://docs.oracle.com/cd/E72372_01/html/E72376/index.html

References

This solution has no attachment