Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-71-1592250.1
Update Date:2018-05-16
Keywords:

Solution Type  Technical Instruction Sure

Solution  1592250.1 :   How to Replace a Sun Server X4-2 Motherboard  


Related Items
  • Sun Server X4-2
  •  
  • Oracle Advanced Support Gateway Server X4-2
  •  
Related Categories
  • PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
  •  




Oracle Confidential PARTNER - Available to partners (SUN).
Reason: FRU CAP

Applies to:

Oracle Advanced Support Gateway Server X4-2 - Version All Versions and later
Sun Server X4-2 - Version All Versions to All Versions [Release All Releases]
x86_64

Goal

How to Replace a Sun Server X4-2 Motherboard.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED?:
Sun Server X4-2 Training

TIME ESTIMATE: 90 minutes

TASK COMPLEXITY: 3

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

PROBLEM OVERVIEW: A Sun Server X4-2 Motherboard needs replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

NOTE:  If you are replacing a system board in an Engineered system, such as Exadata or Exalogic, please use the CAP specific to the Engineered systems since they require additional configuration.
How to replace a Motherboard in a server in Exadata Database Machine [X3-2/X3-8] (Doc ID 1560852.1)
How to Replace a Motherboard in a Oracle Exalogic Elastic Cloud Compute Node (X2-2, X3-2, or X4-2) (Doc ID 1496559.1)

Note: The removal/insertion CPU tool is new for the Sandy Bridge and Ivy Bridge product lines.  If you have not used this new tool before please make yourself familiar before attempting to use onsite.  The tool is not intuitive so reference the service manual before attempting this service action.

If the system is still up and functioning, customer should perform an orderly and graceful shutdown of applications and OS.  Then power off the server and remove the AC power cords from the system.

A data backup is not a prerequisite but is a wise precaution.

Animation videos are available to demonstrate the CPU replacement process for these servers. please refer to the following links:

Please also refer to the "Oracle Sun x86 Processor Replacement Guide" for detailed instruction on CPU replacements and tool usage.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

1.Backup the ILOM configuration.

  1. As long as the system's ILOM is functional and not the reason for the motherboard replacement the ILOM configuration should be backed up so that it may be restored to the new replacement board. If the ILOM is not accessible and cannot be backed up the FE will need to re-configure the ILOM manually after the motherboard replacement.
  2. To backup the settings using the ILOM web browser expand the "ILOM Administration" section in the left column and then select "Configuration Management". In the main panel you may then backup the ILOM settings.
  3. To backup the settings using the ILOM cli you must use scp to copy the file to an available system on the admin network (or your laptop if you connect it directly to the ILOM net management port and configure it for the same network). Use the following commands substituting the appropriate values for the login name, password, ipaddress and file destination. The passphrase should be set to the ILOM's password.
    -> cd /SP/config
    -> set passphrase=ilompassswd
    -> set dump_uri=scp://username:password@IPaddress/tmp/SP.config

2.While still logged into the ILOM check the PSNC values and sync them if needed.

  1. To avoid mismatched PSNC values causing a failure after a system board replacement the fruid data should be confirmed to have matching data in at least the Primary (DBP0) and Backup2 (PS0) containers so that the system board will have it's container updated automatically after replacement. Go into restricted mode and use the showpsnc command to check this.  
    -> set SESSION mode=restricted

    WARNING: The "Restricted Shell" account is provided solely
    to allow Services to perform diagnostic tasks.

    [(restricted_shell) sunx4-2-sp:~]$showpsnc
    Primary: fruid:///SYS/DBP0
    Backup 1: fruid:///SYS/MB
    Backup 2: fruid:///SYS/PS0

    Element           | Primary           | Backup1           | Backup2           
    ------------------+-------------------+-------------------+-------------------
    PPN                 32180288+1+1        32180288+1+1        32180288+1+1      
    PSN                 1234NML12V          1234NML12V          1234NML12V        
    Product Name        SUN SERVER X4-2     SUN SERVER X4-2     SUN SERVER X4-2 
    [(restricted_shell) sunx4-2-sp:~]$exit
     
  2. The above example shows a system with all three containers properly in sync. If the output from the system does not show all of the containers with matching values then you should reset the SP and then re-check the values again. An ILOM reset will attempt to auto-populate the matching values if one container is out of sync.  
    -> reset /SP
    Are you sure you want to reset /SP (y/n)? y
    Performing reset on /SP
     
  3. After an ILOM reset if the Primary and Backup2 containers match then proceed with the following steps to replace the system board. If these two containers do not match then DO NOT proceed with the replacement yet.
  4. If the containers do not match you will need to use the copypsnc command from service or escalation mode to copy the data from the good container so that the Primary and Backup 2 containers match (Backup 1 is the MB and we are about to replace this so it is not as important at this step). If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to Replace a Sun Server X4-2 Storage Disk Backplane (Doc ID 1592273.1)" and contact the TSC if needed.
  5. After the PSNC data in the Primary and Backup2 containers has been confirmed to match proceed with the following steps.

3. Prepare the server for service.

  1. Power off the server and disconnect the power cords from the power supplies.
  2. Extend the server to the maintenance position in the rack.
  3. Attach an anti-static wrist strap.
  4. Remove the top cover.

4. Remove all of the Fan Modules

  1. Open the Server Fan Door. Slide the fan door latches forward and swing the door up to the open position.
  2. Using your thumb and forefinger, grasp the fan module in the finger recesses located in the plastic between the fans.
  3. Lift the fan module straight up and out of the chassis.

5. Remove the Power Supplies

  1. Gain access to the rear of the server where the power supplies are located.
  2. Release the cable management arm (CMA).If one is installed, Press and hold the CMA release tab.
  3. Rotate the cable management arm out of the way so that you can access the power supply.
  4. Disconnect the power cord from each power supply.
  5. Label each power supply with the slot number from which it was removed (PS0, PS1) so that they can be re-installed to the same slots so that the FRU TLI data is not lost.
  6. For each of the power supplies grasp the power supply handle and push the power supply latch to the left.
  7. Pull the power supply out of the chassis.

6. Remove the PCIe risers and PCIe cards.

  1. If any PCIe cards are installed in the risers, disconnect any external or internal cables from the cards. Note the cable locations and their order to ensure they are reinstalled in their proper locations.
  2. For each of the PCIe risers in Slots 1 and 2 lift the green-tabbed latch on the rear of the server's chassis next to the PCIe slot to release the PCIe card's rear bracket.
  3. Lift the riser release lever with one hand and use your other hand to remove the riser from the motherboard
  4. place the riser on an anti-static mat and make a note of the slot in which the PCIe riser was originally installed so that it can be reinstalled into it's proper slot.
  5. For the Slot 3 and 4 PCIe riser open the green-tabbed latch located on the rear of the server's chassis next to the PCIe slot 3 to release the PCIe card holding bracket.
  6. To release the riser from the motherboard connector, lift the riser's green-tabbed release lever to the open position.
  7. Slide the plastic PCIe card retainer, which is mounted on the side of the chassis, forward to release the card(s) installed in the riser
  8. Grasp the riser with both hands and remove it from the server
  9. Disconnect the SAS storage drive (HDD) cables from the internal HBA card installed in PCIe slot 4 Note the cable locations and their order to ensure they are reinstalled in their proper locations.
  10. place the riser on an anti-static mat and make a note of the slot in which the PCIe riser was originally installed so that it can be reinstalled into it's proper slot.

7. Remove the Motherboard from the Chassis.

  1. Disconnect the disk backplane power cable from the motherboard
  2. Disconnect disk backplane LED cable from the motherboard.
  3. Disconnect the front indicator module (FIM) cable from the motherboard.
  4. If the server has a DVD drive, Disconnect the DVD drive cable from the motherboard and carefully lift the DVD drive cable out of the cable trough.
  5. Carefully lift the SAS cables that you disconnected from the internal HBA out of the cable trough.
  6. Remove the server's mid-wall by loosening the four green captive screws that secure the mid-wall to the server's chassis. Then lift the mid-wall slightly up and toward the front of the server to disengage it from the mushroom standoffs that are located on the server's chassis sidewall (one on each end of the mid-wall).
  7. Remove the motherboard from the server's chassis by grasping the middle of the air duct in the front and back and slide the motherboard toward the front of the server then lift it slightly to disengage the six mushroom standoffs that are located on the server's chassis under the motherboard.
  8. Lift the motherboard out of the server's chassis and place it on an anti-static mat, and next to the replacement motherboard.
  9. When removing the motherboard from the chassis take care that the plastic lightpipe that is connected to the rear locate led/button stays connected. This lightpipe can become disconnected when removing the board from the chassis and care should be taken to make sure the board is returned with the lightpipe properly attached. (the new replacement board should have one already connected to the board)
  10. Remove the following reusable components from the motherboard and install them on the replacement motherboard.
    1. Internal USB flash drives (remove plastic cover by CPU location P1 for access).
    2. Make sure to install each USB flash drive in the same orientation to ensure that it is properly positioned in the USB port.
    3. DDR3 DIMMs. Perform a one-to-one replacement of the DIMMs from the current motherboard to the new motherboard to reduce the possibility of the DIMMs being installed into the wrong slots. If the DIMMs are not populated correctly the server performance may suffer and some DIMMs might not be used.
    4. To remove the DIMMs rotate both DIMM socket ejectors outward as far as they will go.
    5. Carefully lift the DIMM straight up to remove it from the socket.
    6. Install the DIMMs in the replacement Motherboard by ensuring that the ejector tabs of the DIMM slots are in the open position.
    7. Then align the notch in the DIMM with the key in the connector.
    8. Push the DDR3 DIMM into the connector until the ejector tabs lock the DIMM in place.

8. Identify the correct CPU removal/insertion tool.

  1. The X4 systems support a 12-core processor that is slightly larger than the other processor types. Due to this size difference a different tool is needed. Motherboard FRUs should ship with both tools and Cpu replacement FRUs should ship with the appropriate tool but it is recommended that the field engineer double-check to avoid the possibility of damage to the cpu or socket due to use of the wrong tool. The tools are color coded for easy identification. For a 12-core processor the tool utilizes a pink sticker to indicate the corner containing pin 1, for all other processor types the tool has a green sticker.
  2. After confirming that the proper tool is available place it upon the antistatic mat until needed.

9. Remove the CPU(s) from the current Motherboard. Perform the following steps for each installed CPU.

  1. Gently press down on the top of the heat sink to counteract the pressure of the captive spring-loaded screws that secure the heat sink to the motherboard.  Loosen the four Phillips captive screws in the heat sink for the faulty CPU.  Using a No. 2 Phillips screwdriver turn the screws counter clockwise alternately one and one half turns until they are fully removed.
  2. To separate the heat sink from the top of the CPU, gently wiggle the heat sink left and right, while pulling upward.  A thin layer of thermal compound separates the heat sink and the CPU.  This compound also acts as an adhesive. Note –Do not allow the thermal compound to contaminate the work space or other components.
  3. Place the heat sink upside down on a flat surface.
  4. Use an alcohol pad to clean the thermal grease from both the bottom of the heat sink and the top of the CPU. Caution –Handle CPU socket pins with extreme care. CPU and socket pins are very fragile.  A light touch can bend the CPU socket pins and damage the board beyond repair.
  5. Disengage both CPU release levers by pushing down and moving it to the side away from the CPU, and then rotating the levers upward starting with the right lever (when viewing server from the front).  Repeat with left lever.
  6. Lift the CPU pressure frame up to the fully open position.
  7. This platform requires a custom removal/insertion CPU tool that comes with the CPU replacement FRU.  Push the removal/insertion tool button in the top center to unlock the tool.  Place the removal/insertion tool on top of the CPU.  Properly position the tool over the CPU socket and lower it into place over the CPU socket.  Ensure that the tool and CPU key corners are aligned.  To properly position the tool over the CPU socket, rotate the tool until the green triangle on the side of the tool is facing the front of the server and it is over the left side of the CPU socket when viewing the server from the front.
  8. Click the tab on the removal/insertion tool to secure the tool to the CPU.  Press the release lever on the tool to release the center button and engage the CPU.  An audible click indicates that the CPU is engaged.
  9. Lift the CPU out of the socket while it is still attached to the removal/insertion tool.  Grasp the tool by its sides and remove it from the server.  Turn the tool upside down and verify that it contains the CPU.
  10. Leave the CPU in the removal/insertion tool and proceed to next step to install it into the new Motherboard.
  11. The replacement motherboard should have shipped with CPU socket covers protecting each of the sockets. After moving the CPUs over to the new motherboard the socket covers should be installed onto the old board so that the sockets are not damaged while being shipped back to the repair depot. Please take care when installing the covers so that the socket pins are not damaged. Note that the covers are installed ON TOP of the CPU pressure frame and NOT underneath it. (previous server designs located these beneath the frame but the X3 servers install these on the top) Installing the cover under the frame WILL damage the socket.

10. Re-install the CPU(s) onto the new Motherboard. Perform the following steps for each CPU.

  1. Ensure that the two CPU socket release levers are in the fully open position and that the CPU pressure frame is in the fully open position.
  2. The CPU removal/insertion tool should have the CPU installed in it from the previous step after removing it from the current motherboard.
  3. Properly position the tool over the CPU socket and lower it into place.  Carefully align the CPU over the CPU socket.  Ensure that the notches on the sides of the CPU align with the keys on the socket.  To properly position the tool in the CPU socket, rotate the tool until the green triangle on the side of the tool is facing the front of the server and it is over the left side of the CPU socket (when viewing server from the front) and lower the tool onto the CPU socket.
  4. Press the center button in the tool down to release the CPU so that it is placed in the CPU socket. Caution – Do not press down on the CPU.  Irreparable damage to the CPU or motherboard might occur from excessive downward pressure.  Do not forcibly seat the CPU into the socket.  Excessive downward pressure might damage the socket pins.
  5. Remove the processor removal/insertion tool.
  6. Visually check the alignment.  Close the pressure frame ensure that the pressure frame sits flat around the periphery of the CPU.
  7. Engage the release levers by rotating them downward and slipping them under the catch left first and then the right one.

11.Install the CPU Heat Sink Assembly.

  1. Use the syringe to apply approximately 0.1 ml of thermal grease to the center of the top of the CPU.  Do not distribute the grease.
  2. Inspect the heat sink for dust and lint. Clean if necessary.
  3. Carefully position the heat sink on the CPU, aligning it with the mounting posts to reduce movement after it makes initial contact with the layer of thermal grease. Caution – When handling the heat sink, be careful not to get the thermal compound grease on your fingers so as not to transfer the grease to other server components.
  4. Lower the heat sink onto the CPU. Caution – Avoid moving the heat sink after it has contacted the top of the CPU.  Too much movement could disturb the layer of thermal compound, causing voids, and leading to ineffective heat dissipation and component damage.
  5. Tighten the Phillips screws alternately one-half turn until they are fully seated.

12. Re-install the Motherboard into the Chassis.

  1. grasp the middle of the air duct in the front and back and tilt the front of the motherboard up slightly and push it into the opening in the rear of the server's chassis.
  2. Lower the motherboard into the server's chassis and slide it to the rear until it engages the six mushroom standoffs located on the server's chassis under the motherboard.
  3. Ensue that the indicators, controls and connectors on the rear of the motherboard fit correctly into the rear of the server's chassis.
  4. Install the server's mid-wall. Ensure that the cables (SAS and DVD cable, if applicable), are running through the opening in the center of the mid-wall.
  5. Position the mid-wall with cables in front of the motherboard so that it engages the mushroom standoffs that are located on the server's chassis sidewall (one for each side of the mid-wall).
  6. Place the FIM cable and the disk backplane power cable along side the mid-wall next to their respective motherboard connectors.
  7. Ensure that FIM cable and the disk backplane power cable are not pinched by the mid-wall and that they run beside the mid-wall and not under it; otherwise, the cables might be damaged.
  8. To secure the mid-wall to the server's chassis, tighten the four green captive screws.
  9. Carefully place the SAS cable(s) that you disconnected from the internal HBA into the cable trough provided by the air duct on the motherboard.
  10. If the server has a DVD drive, carefully place the DVD drive cable into the cable trough provided by the air duct on the motherboard and reconnect the DVD drive cable to the motherboard.
  11. Reconnect the front indicator module (FIM) cable to the motherboard.
  12. Reconnect disk backplane LED cable to the motherboard.
  13. Reconnect the disk backplane power cable to the motherboard.

13. Re-install the PCIe risers and PCIe cards.

  1. For each of the PCIe slot 1 and 2 risers, raise the PCIe riser release lever (marked with a green tab) to the open (up) position
  2. Making sure to replace the riser into the same position from which it was removed gently press the riser into the motherboard connector until it seats and press the green-tabbed, riser release lever to the closed (down) position.
  3. If there is a PCIe card installed in the riser, reconnect any external or internal cables to the card.
  4. Close the green-tabbed latch on the rear of the server's chassis next to the applicable PCIe slot to secure the PCIe card's rear bracket to the server's chassis. If the riser does not have a PCIe card installed, install a PCIe slot filler panel and close the green-tabbed latch to secure filler panel.
  5. For the PCIe slots 3 and 4 riser, reconnect the SAS cable(s) to the internal HBA card.Be sure to connect the SAS cable for storage drives 0 through 3 (HDDs 0-3) to the connector that is farther from the riser in which the HBA card is installed.
  6. Raise the PCIe riser green-tabbed release lever to the open (up) position and gently press the riser into the motherboard connector until it seats.
  7. Ensure that the rear bracket on the internal HBA card in PCIe slot 4 is connected to the slot in the server's chassis side wall. If the bracket is not connected, remove the riser and reposition it so that the rear bracket connects to the side wall, then gently press the riser into the motherboard connector.
  8. Slide the plastic PCIe card retainer that is mounted on the side of the chassis toward the back of the server to secure the card(s) installed in the riser.
  9. Press the green-tabbed release lever on the PCIe riser to the close (down) position.
  10. To secure the PCIe card's rear bracket to the server, close the green-tabbed latch on the rear of the server's chassis.
  11. If there is a PCIe card installed in slot 3 of the riser, reconnect any external or internal cables to the card. If the riser does not have a PCIe card installed in slot 3, install a PcIe filler panel and close the green-tabbed latch to secure the PCIe slot filler panel.

14. Re-install the Power Supplies and Fans

  1. Make sure to reinstall the power supplies into the same slots from which they were removed so that the FRU TLI data is preserved and the serial number information will be auto-updated onto the new system board.
  2. For each power supply align the power supply with the empty power supply bay, then Slide the power supply into the bay until it is fully seated.
  3. Open the fan door and install the fan modules into the server.
  4. Press down on the fan module and apply firm pressure to fully seat the fan module.
  5. Close the fan door.

15. Return the Server to operation

  1. Replace the top cover
  2. Remove any anti-static measures that were used.
  3. Return the server to it's normal operating position within the rack.
  4. Re-install the AC power cords and any data cables that were removed.
  5. Power on server. Verify that the Power/OK indicator led lights steady on.

16. Restore ILOM settings from the backup created in the first step.

  1. To restore the settings using the ILOM web browser expand the "ILOM Administration" section in the left column and then select "Configuration Management". In the main panel you may then restore the ILOM settings.
  2. To restore the settings using the ILOM cli you must use scp to copy the file from an available system on the admin network should be the same system that was used for the backup in step 1. Use the following commands substituting the appropriate values for the login name, password, ipaddress and file destination. The passphrase should be set to the ILOM's password.
    -> cd /SP/config
    -> set passphrase=ilompassswd
    -> set load_uri=scp://username:password@IPaddress/tmp/SP.config
  3. If the ILOM was not able to be backed up then the FE will need to manually configure the ILOM settings. Use the commands in the example below to set the network, hostname, system identifier, timezone and date/time. Substitute the appropriate values for the customer system. The ntp settings only need to be set if the customer is using an ntp server.
    -> set /SP/network pendingipaddress=192.168.1.111
    -> set /SP/network pendingipnetmask=255.255.255.0
    -> set /SP/network pendingipgateway=192.168.1.1
    -> set /SP/network pendingipdiscovery=static
    -> set /SP/network commitpending=true
    -> set /SP/network state=enabled
    -> set /SP hostname=hostname-sp
    -> set /SP/clock timezone=EST
    -> set /SP/clock datetime=MMDDhhmmYYYY
    -> set /SP/clients/ntp/server/1 address=ntp_server_ip_address
    -> set /SP/clock usentpserver=enabled

17. Check and set the system serial number/fruid data if needed. 

  1. login to the ILOM as root and then enter the restricted shell to check the psnc values. Follow the example below to enter restricted shell and use the showpsnc command-

    -> set SESSION mode=restricted

    WARNING: The "Restricted Shell" account is provided solely
    to allow Services to perform diagnostic tasks.

    [(restricted_shell) sunx4-2-sp:~]$showpsnc
    Primary: fruid:///SYS/DBP0
    Backup 1: fruid:///SYS/MB
    Backup 2: fruid:///SYS/PS0

    Element           | Primary           | Backup1           | Backup2           
    ------------------+-------------------+-------------------+-------------------
    PPN                 32180288+1+1        32180288+1+1        32180288+1+1      
    PSN                 1234NML12V          1234NML12V          1234NML12V        
    Product Name        SUN SERVER X4-2     SUN SERVER X4-2     SUN SERVER X4-2 
    [(restricted_shell) sunx4-2-sp:~]$exit

  2. With a motherboard replacement the system should be able to automatically copy the psnc values from the primary and backup2 containers as long as they match. (if more than one part was replaced at the same time this may not happen)
  3. As long as all of the psnc values match and are correct as in the above example this step is done. However if there was an issue during the replacement which caused the psnc values not to propagate automatically then this will need to be fixed manually by using the copypsnc command from service or escalation mode. If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to Replace a Sun Server X4-2 Storage Disk Backplane (Doc ID 1592273.1)" and contact the TSC if needed.

 18. The end user of the server should be made aware that the system board replacement will result in the on-board Ethernet interfaces containing different MAC addresses when compared to the original system board. If the system OS is configured using the MAC addresses from the original system board then the system administrator should take the necessary steps to update the OS configuration to use the MAC addresses on the newly installed board.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Boot up system and verify full functionality

REFERENCE INFORMATION:
Sun Server X4-2 Documentation

http://docs.oracle.com/cd/E36975_01/index.html


Attachments
This solution has no attachment
  Copyright © 2018 Oracle, Inc.  All rights reserved.
 Feedback