Sun Server X3-2L
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

Applies to:

Goal

How to Replace a Sun Server X3-2L (Sun Fire X4270 M3) Motherboard.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE FIELD ENGINEER/ADMINISTRATOR NEED?:
Sun Server X3-2L (Sun Fire X4270 M3) Training

TIME ESTIMATE: 90 minutes

TASK COMPLEXITY: 3

FIELD ENGINEER/ADMINISTRATOR INSTRUCTIONS:

PROBLEM OVERVIEW: A Sun Server X3-2L (Sun Fire X4270 M3) Motherboard needs replacement

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

If the system is still up and functioning, customer should perform an orderly and graceful shutdown of applications and OS.  Then power off the server and remove the AC power cords from the system.

A data backup is not a prerequisite but is a wise precaution.

Please also refer to the "Oracle Sun x86 Processor Replacement Guide" for detailed instruction on CPU replacements and tool usage.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

1.Backup the ILOM configuration.

1. As long as the system's ILOM is functional and not the reason for the motherboard replacement the ILOM configuration should be backed up so that it may be restored to the new replacement board. If the ILOM is not accessible and cannot be backed up the FE will need to re-configure the ILOM manually after the motherboard replacement.
2. To backup the settings using the ILOM web browser expand the "ILOM Administration" section in the left column and then select "Configuration Management". In the main panel you may then backup the ILOM settings.
3. To backup the settings using the ILOM cli you must use scp to copy the file to an available system on the admin network (or your laptop if you connect it directly to the ILOM net management port and configure it for the same network). Use the following commands substituting the appropriate values for the login name, password, ipaddress and file destination. The passphrase should be set to the ILOM's password.
   -> cd /SP/config
   -> set passphrase=ilompassswd
   -> set dump_uri=scp://username:password@IPaddress/tmp/SP.config

2. While still logged into the ILOM check the PSNC values and sync them if needed.

1. To avoid mismatched PSNC values causing a failure after a system board replacement the fruid data should be confirmed to have matching data in at least the Primary (DBP0) and Backup2 (PS0) containers so that the system board will have it's container updated automatically after replacement. Go into restricted mode and use the showpsnc command to check this.  
   -> set SESSION mode=restricted
   
   WARNING: The "Restricted Shell" account is provided solely
   to allow Services to perform diagnostic tasks.
   
   
   [(escalation_mode) sunx3-2l-sp:~]# showpsnc
   Primary: fruid:///SYS/DBP0
   Backup 1: fruid:///SYS/MB
   Backup 2: fruid:///SYS/PS0
   
   Element           | Primary           | Backup1           | Backup2           
   ------------------+-------------------+-------------------+-------------------
   PPN                 30525574+1+1        30525574+1+1        30525574+1+1       
   PSN                 1234FML123          1234FML123          1234FML123         
   Product Name        SUN FIRE X4270 M3   SUN FIRE X4270 M3   SUN FIRE X4270 M3 
   [(escalation_mode) sunx3-2l-sp:~]# exit
    
2. The above example shows a system with all three containers properly in sync. If the output from the system does not show all of the containers with matching values then you should reset the SP and then re-check the values again. An ILOM reset will attempt to auto-populate the matching values if one container is out of sync.  
   -> reset /SP
   Are you sure you want to reset /SP (y/n)? y
   Performing reset on /SP
    
3. After an ILOM reset if the Primary and Backup2 containers match then proceed with the following steps to replace the system board. If these two containers do not match then DO NOT proceed with the replacement yet.
4. If the containers do not match you will need to use the copypsnc command from service or escalation mode to copy the data from the good container so that the Primary and Backup 2 containers match (Backup 1 is the MB and we are about to replace this so it is not as important at this step). If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to Replace a Sun Server X3-2L(X4270M3) Storage Disk Backplane for Eight-Drive and Twelve-Drive Systems (Doc ID 1496009.1)" and contact the TSC if needed.
5. After the PSNC data in the Primary and Backup2 containers has been confirmed to match proceed with the following steps.

3. Prepare the server for service.

1. Power off the server and disconnect the power cords from the power supplies.
2. Extend the server to the maintenance position in the rack.
3. Attach an anti-static wrist strap.
4. Remove the top cover.
5. Remove the Air Baffle

4. Remove all of the Fan Modules

1. Open the Server Fan Door. Slide the fan door latches forward and swing the door up to the open position.
2. Using your thumb and forefinger, loosen the captive screw that secures the faulty fan module in the chassis (use a No. 2 Phillips screwdriver if it is too tight to loosen with your fingers).
3. Grasp both the captive screw and the opposite end of the module and lift the fan module straight up and out of the chassis (do not rock it back and forth), and set it aside on an antistatic mat.

5. Remove the Power Supplies

1. Gain access to the rear of the server where the power supplies are located.
2. Release the cable management arm (CMA).If one is installed, Press and hold the CMA release tab.
3. Rotate the cable management arm out of the way so that you can access the power supply.
4. Disconnect the power cord from each power supply.
5. Label each power supply with the slot number from which it was removed (PS0, PS1) so that they can be re-installed to the same slots so that the FRU TLI data is not lost.
6. For each of the power supplies grasp the power supply handle and push the power supply latch to the left.
7. Pull the power supply out of the chassis.

6. Remove the Storage Drive Cables and SAS expander from a SAS/SATA configuration.

1. If you are removing the SAS/SATA cables from a server with twenty-four 2.5-inch storage drives, remove the server's front fan assembly door cover.
2. Using a No. 1 Phillips screwdriver, remove the two screws (one on each side of the chassis) that secure the fan assembly door.
3. Slide the fan assembly door toward the rear of the server, and lift and remove the door from the chassis.
4. Disconnect SAS/SATA cables from the SAS expander module making note of their locations so that they may be re-connected in the same positions.
5. To disconnect the HBA cable and the SAS/SATA cables from the SAS expander module, press the latch, and then pull out to disengage each cable from the SAS expander module.
6. To disconnect the SAS/SATA cables from the front storage drive backplane and the SAS expander module, press the latch, and then pull out to disengage the cables from each storage drive backplane.
7. If they are installed disconnect the rear storage drive SAS cables from the SAS expander module, press the latch, and then pull up to disengage the disk drive SAS drive cable from the SAS expander module.
8. Remove the SAS expander module from the chassis.
9. Locate and loosen the four black push pins that secure the SAS expander module to the server's chassis. Loosen each push pin by pulling up on the pin until it clicks into the open position
10. Grasp the sides of the SAS expander module, and gently lift the module to disengage the module from the chassis connector
11. Place the SAS expander module on an antistatic mat.

7. Remove any installed PCIe cards.

1. For each PCIe card make a note of where the PCIe cards are installed so that they can be re-installed in the same locations.
2. Unplug all data cables from the PCIe card
3. Rotate the PCIe card locking mechanism, and then lift up on the PCIe card to disengage it from the motherboard connectors.
4. Place the PCIe card on an antistatic mat.

8. Remove the Motherboard from the Chassis.

1. Disconnect the ribbon cables from the left front I/O module and right front I/O modules.
2. Disconnect the LED cable between the motherboard to the disk backplane.
3. Disconnect the signal cable from the server intrusion switch.
4. Remove the server's mid-wall by using a No. 2 Phillips screwdriver, remove the two screws on each side of the chassis that secure the mid-wall to the chassis.
5. Using a No. 2 Phillips screwdriver, loosen the four green captive screws that secure the chassis mid-wall to the bottom of the chassis.
6. Lift the mid-wall out of the chassis
7. Remove the motherboard from the server's chassis by carefully sliding the motherboard forward, and then lifting it out of the chassis.
8. Place it on an anti-static mat, and next to the replacement motherboard.
9. Remove the following reusable components from the motherboard and install them on the replacement motherboard.
   1. Internal USB flash drives, remove by grasping the flash drive and pulling straight up.
   2. Make sure to install each USB flash drive in the same orientation to ensure that it is properly positioned in the USB port.
   3. DDR3 DIMMs. Perform a one-to-one replacement of the DIMMs from the current motherboard to the new motherboard to reduce the possibility of the DIMMs being installed into the wrong slots. If the DIMMs are not populated correctly the server performance may suffer and some DIMMs might not be used.
   4. To remove the DIMMs rotate both DIMM socket ejectors outward as far as they will go.
   5. Carefully lift the DIMM straight up to remove it from the socket.
   6. Install the DIMMs in the replacement Motherboard by ensuring that the ejector tabs of the DIMM slots are in the open position.
   7. Then align the notch in the DIMM with the key in the connector.
   8. Push the DDR3 DIMM into the connector until the ejector tabs lock the DIMM in place.

9. Remove the CPU(s) from the current Motherboard. Perform the following steps for each installed CPU.

1. Gently press down on the top of the heat sink to counteract the pressure of the captive spring-loaded screws that secure the heat sink to the motherboard.  Loosen the four Phillips captive screws in the heat sink for the faulty CPU.  Using a No. 2 Phillips screwdriver turn the screws counter clockwise alternately one and one half turns until they are fully removed.
2. To separate the heat sink from the top of the CPU, gently wiggle the heat sink left and right, while pulling upward.  A thin layer of thermal compound separates the heat sink and the CPU.  This compound also acts as an adhesive. Note –Do not allow the thermal compound to contaminate the work space or other components.
3. Place the heat sink upside down on a flat surface.
4. Use an alcohol pad to clean the thermal grease from both the bottom of the heat sink and the top of the CPU. Caution –Handle CPU socket pins with extreme care. CPU and socket pins are very fragile.  A light touch can bend the CPU socket pins and damage the board beyond repair.
5. Disengage both CPU release levers by pushing down and moving it to the side away from the CPU, and then rotating the levers upward starting with the right lever (when viewing server from the front).  Repeat with left lever.
6. Lift the CPU pressure frame up to the fully open position.
7. This platform requires a custom removal/insertion CPU tool that comes with the CPU replacement FRU.  Push the removal/insertion tool button in the top center to unlock the tool.  Place the removal/insertion tool on top of the CPU.  Properly position the tool over the CPU socket and lower it into place over the CPU socket.  Ensure that the tool and CPU key corners are aligned.  To properly position the tool over the CPU socket, rotate the tool until the green triangle on the side of the tool is facing the front of the server and it is over the left side of the CPU socket when viewing the server from the front.
8. Click the tab on the removal/insertion tool to secure the tool to the CPU.  Press the release lever on the tool to release the center button and engage the CPU.  An audible click indicates that the CPU is engaged.
9. Lift the CPU out of the socket while it is still attached to the removal/insertion tool.  Grasp the tool by its sides and remove it from the server.  Turn the tool upside down and verify that it contains the CPU.
10. Leave the CPU in the removal/insertion tool and proceed to next step to install it into the new Motherboard.
11. The replacement motherboard should have shipped with CPU socket covers protecting each of the sockets. After moving the CPUs over to the new motherboard the socket covers should be installed onto the old board so that the sockets are not damaged while being shipped back to the repair depot. Please take care when installing the covers so that the socket pins are not damaged. Note that the covers are installed ON TOP of the CPU pressure frame and NOT underneath it. (previous server designs located these beneath the frame but the X3 servers install these on the top) Installing the cover under the frame WILL damage the socket.

10. Re-install the CPU(s) onto the new Motherboard. Perform the following steps for each CPU.

1. Ensure that the two CPU socket release levers are in the fully open position and that the CPU pressure frame is in the fully open position.
2. The CPU removal/insertion tool should have the CPU installed in it from the previous step after removing it from the current motherboard.
3. Properly position the tool over the CPU socket and lower it into place.  Carefully align the CPU over the CPU socket.  Ensure that the notches on the sides of the CPU align with the keys on the socket.  To properly position the tool in the CPU socket, rotate the tool until the green triangle on the side of the tool is facing the front of the server and it is over the left side of the CPU socket (when viewing server from the front) and lower the tool onto the CPU socket.
4. Press the center button in the tool down to release the CPU so that it is placed in the CPU socket. Caution – Do not press down on the CPU.  Irreparable damage to the CPU or motherboard might occur from excessive downward pressure.  Do not forcibly seat the CPU into the socket.  Excessive downward pressure might damage the socket pins.
5. Remove the processor removal/insertion tool.
6. Visually check the alignment.  Close the pressure frame ensure that the pressure frame sits flat around the periphery of the CPU.
7. Engage the release levers by rotating them downward and slipping them under the catch left first and then the right one.

11.Install the CPU Heat Sink Assembly.

1. Use the syringe to apply approximately 0.1 ml of thermal grease to the center of the top of the CPU.  Do not distribute the grease.
2. Inspect the heat sink for dust and lint. Clean if necessary.
3. Carefully position the heat sink on the CPU, aligning it with the mounting posts to reduce movement after it makes initial contact with the layer of thermal grease. Caution – When handling the heat sink, be careful not to get the thermal compound grease on your fingers so as not to transfer the grease to other server components.
4. Lower the heat sink onto the CPU. Caution – Avoid moving the heat sink after it has contacted the top of the CPU.  Too much movement could disturb the layer of thermal compound, causing voids, and leading to ineffective heat dissipation and component damage.
5. Tighten the Phillips screws alternately one-half turn until they are fully seated.

12. Re-install the Motherboard into the Chassis.

1. Carefully lift and place the motherboard assembly into the chassis. Tilt the motherboard to the right side to fit under the power supply assembly, level the motherboard and place it into the server chassis, then slide it to the rear of the server to engage the mushroom standoffs.
2. Install the mid-wall into the chassis.
3. Lift and place the mid-wall into the chassis.
4. Using a No. 2 Phillips screwdriver, tighten the four green captive screws that secure the mid-wall to the bottom of the chassis.
5. Using a No. 2 Phillips screwdriver, insert and tighten the two screws on each side of the chassis to secure the mid-wall in the chassis.
6. Reconnect the server intrusion switch cable to the motherboard.
7. Reconnect the signal cable from the storage drive backplane to the motherboard.
8. Reconnect the ribbon cables from left and right LED/USB indicator modules to the motherboard.

13. Re-install the PCIe cards.

1. Make sure to re-install the cards into the same locations from which they were removed.
2. If necessary, remove the PCIe filler panel from the slot.
3. Insert the PCIe card into the correct slot, and rotate the PCIe locking mechanism to secure the PCIe card in place.
4. Reconnect the cables to the PCIe card that you unplugged during the removal procedure.

14. Re-install the Storage Drive Cables and SAS expander from a SAS/SATA configuration.

1. Install the SAS expander module.
2. Lower the SAS expander module into the server chassis, and gently push down on the module to engage the chassis connector.
3. Push down on the four black push pins to secure the SAS expander module to the server's chassis.
4. Connect the HBA cable and SAS cables to the SAS expander module
5. Route the SAS cables from the disk backplane between the fan modules and between processors and DIMMs in the cable trough.
6. If present Install the cable connectors into the rear-mounted storage drive backplane.To reconnect the rear-mounted storage drive SAS cables to the SAS expander module, plug each cable into its top SAS expander module connector until you hear an audible click.
7. Making sure to reconnect the cables in the same order as they were previously installed reconnect the SAS/SATA cables to the SAS expander module. To reconnect the HBA cable and the SAS/SATA cables to the SAS expander module, plug each cable into its SAS expander module connector until you hear an audible click.
8. To reconnect the front storage drive SAS/SATA cables to the SAS expander module, plug each cable into its SAS expander module connector until you hear and audible click
9. Place the cables into cable tie-downs, and secure them with the tie-downs

15. Re-install the Power Supplies and Fans

1. Make sure to reinstall the power supplies into the same slots from which they were removed so that the FRU TLI data is preserved and the serial number information will be auto-updated onto the new system board.
2. For each power supply align the power supply with the empty power supply bay, then Slide the power supply into the bay until it is fully seated.
3. Open the fan door and install the fan modules into the server.
4. Press down on the fan module and apply firm pressure to fully seat the fan module.
5. Close the fan door.

16. Return the Server to operation

1. Replace the Air Baffle
2. Replace the top cover
3. Remove any anti-static measures that were used.
4. Return the server to it's normal operating position within the rack.
5. Re-install the AC power cords and any data cables that were removed.
6. Power on server. Verify that the Power/OK indicator led lights steady on.

17. Restore ILOM settings from the backup created in the first step.

1. To restore the settings using the ILOM web browser expand the "ILOM Administration" section in the left column and then select "Configuration Management". In the main panel you may then restore the ILOM settings.
2. To restore the settings using the ILOM cli you must use scp to copy the file from an available system on the admin network should be the same system that was used for the backup in step 1. Use the following commands substituting the appropriate values for the login name, password, ipaddress and file destination. The passphrase should be set to the ILOM's password.
   -> cd /SP/config
   -> set passphrase=ilompassswd
   -> set load_uri=scp://username:password@IPaddress/tmp/SP.config
3. If the ILOM was not able to be backed up then the FE will need to manually configure the ILOM settings. Use the commands in the example below to set the network, hostname, system identifier, timezone and date/time. Substitute the appropriate values for the customer system. The ntp settings only need to be set if the customer is using an ntp server.
   
   -> set /SP/network pendingipaddress=192.168.1.111
   -> set /SP/network pendingipnetmask=255.255.255.0
   -> set /SP/network pendingipgateway=192.168.1.1
   -> set /SP/network pendingipdiscovery=static
   -> set /SP/network commitpending=true
   -> set /SP/network state=enabled
   -> set /SP hostname=hostname-sp
   -> set /SP/clock timezone=EST
   -> set /SP/clock datetime=MMDDhhmmYYYY
   -> set /SP/clients/ntp/server/1 address=ntp_server_ip_address
   -> set /SP/clock usentpserver=enabled 

18. Check and set the system serial number/fruid data if needed. 

1. login to the ILOM as root and then enter the restricted shell to check the psnc values. Follow the example below to enter restricted shell and use the showpsnc command-
   -> set SESSION mode=restricted
   
   WARNING: The "Restricted Shell" account is provided solely
   to allow Services to perform diagnostic tasks.
   
   
   [(escalation_mode) sunx3-2l-sp:~]# showpsnc
   Primary: fruid:///SYS/DBP0
   Backup 1: fruid:///SYS/MB
   Backup 2: fruid:///SYS/PS0
   
   Element           | Primary           | Backup1           | Backup2           
   ------------------+-------------------+-------------------+-------------------
   PPN                 30525574+1+1        30525574+1+1        30525574+1+1       
   PSN                 1234FML123          1234FML123          1234FML123         
   Product Name        SUN FIRE X4270 M3   SUN FIRE X4270 M3   SUN FIRE X4270 M3 
   [(escalation_mode) sunx3-2l-sp:~]# exit
2. With a Motherboard replacement the system should be able to automatically copy the psnc values from the primary and backup2 containers as long as they match. (if more than one part was replaced at the same time this may not happen)
3. As long as all of the psnc values match and are correct as in the above example this step is done. However if there was an issue during the replacement which caused the psnc values not to propagate automatically then this will need to be fixed manually by using the copypsnc command from service or escalation mode. If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to Replace a Sun Server X3-2L(X4270M3) Storage Disk Backplane for Eight-Drive and Twelve-Drive Systems (Doc ID 1496009.1)" and contact the TSC if needed.

 19. The end user of the server should be made aware that the system board replacement will result in the on-board Ethernet interfaces containing different MAC addresses when compared to the original system board. If the system OS is configured using the MAC addresses from the original system board then the system administrator should take the necessary steps to update the OS configuration to use the MAC addresses on the newly installed board.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Boot up system and verify full functionality

REFERENCE INFORMATION:
Sun Server X3-2L Documentation

http://docs.oracle.com/cd/E23393_01/index.html

This solution has no attachment