Netra Server X5-2
 

PLA-Support>Sun Systems>Sun_Other>Sun Collections>SN-OTH: x64-CAP VCAP
 

Applies to:

Goal

How to Replace the PDB on an Netra Server X5-2

Solution

CAP PROBLEM OVERVIEW: Power Distribution Board (PDB) replacement

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE ENGINEER NEED:
Netra Server X5-2 Training.

TIME ESTIMATE: 30 minutes

TASK COMPLEXITY: 3-FRU

FIELD ENGINEER INSTRUCTIONS

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

If the system is still up and functioning, customer should perform an orderly and graceful shutdown of applications and OS.  A data backup is not a prerequisite but is a wise precaution.

Removing the PDB is a cold-service operation. You must power off the server and and remove the AC power cords from the system before you remove the PDB

1. Consider your first step.

•  If you have not prepared for service, do so now. See “Preparing for Service” page 61 in Service Manual.

2. Review this information in Service Manual and complete these prerequisites in this order.

• Remove the front Air Filter on page 89
• Remove Power Supply on page 112
  • Remove both power supplies
• Remove a Fan Module on page 122
  • Remove all fan modules.
• Remove the Fan Tray on page 80
• Rotate the Drive Cage to the Service Position on page 81
• Remove HBA Super Cap
• Remove the Air Duct on page 83
• Remove the PSU Duct on page 87
• Remove PCIe card on page 129
• Remove the USB Flash Drive on page 140

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

Servicing the PDB Assembly:

http://docs.oracle.com/cd/E53596_01/html/E53597/nbpov.z40000f22025954.html#scrolltoc

1. Log into the ILOM check the fruid container values and sync them if needed.

1. To avoid mismatched fruid values causing a failure after a PDB replacement the fruid data should be confirmed to have matching data in at least the Primary (DBP) and Backup1 (MB) containers so that the PDB will have it's container updated automatically after replacement. Go into restricted mode and use the showpsnc command to check this.  
   -> set SESSION mode=restricted
   
   WARNING: The "Restricted Shell" account is provided solely
   to allow Services to perform diagnostic tasks.
   
   [(restricted_shell) ORACLESP-NX5-2:~]# showpsnc
   Primary: fruid:///SYS/DBP
   Backup 1: fruid:///SYS/MB
   Backup 2: fruid:///SYS/PDB
   
   Element           | Primary           | Backup1           | Backup2           
   ------------------+-------------------+-------------------+-------------------
   PPN                 NSX5-2              NSX5-2              NSX5-2      
   PSN                 1450NMB006          1450NMB006          1450NMB006         
   Product Name        NETRA SERVER X5-2   NETRA SERVER X5-2   NETRA SERVER X5-2  
   
   
   [(restricted_shell) ORACLESP-NX5-2:~]# exit
    
2. The above example shows a system with all three containers properly in sync. If the output from the system does not show all of the containers with matching values then you should reset the SP and then re-check the values again. An ILOM reset will attempt to auto-populate the matching values if one container is out of sync.  
   -> reset /SP
   Are you sure you want to reset /SP (y/n)? y
   Performing reset on /SP
    
3. After an ILOM reset if the Primary and Backup1 containers match then proceed with the following steps to replace the motherboard. If these two containers do not match then DO NOT proceed with the replacement yet.
4. If the containers do not match you will need to use the copypsnc command from service or escalation mode to copy the data from the good container so that the Primary and Backup1 containers match (Backup2 is the PDB and we are about to replace this so it is not as important at this step). If you are unfamiliar with this process and require assistance please reference the steps for using copypsnc to fix the serial number detailed in the "How to update product serial number on systems which implement TLI functionality (Doc ID 1280913.1)" and contact the TSC if needed. How to access service mode and escalation mode on ILOM 3.x and later platforms (Doc ID 1019946.1)
5. After the fruid data in the Primary and Backup1 containers have been confirmed to match proceed with the following steps.

2. Prepare the server for service.

1. Power off the server and disconnect the power cords from the power supplies.
2. Remove  the server from the rack.
   1. 4-Post, 19-inch Hardmount Rack Instructions
      1. Support the weight of the server for removal.
      2. Loosen the center captive screw on the left and right rear support brackets.
      3. Remove the two screws securing each hardmount bracket at the front of the rack.
      4. Lift the server from the rack and set it aside on an antistatic mat.
   2. 2-Post, 19-inch Hardmount Rack
      1. Support the weight of the server for removal.
      2. Remove the two screws that secure each rear plate to the back of the post.
      3. Loosen the two screws that secure each rear plate to the eyelets of the side bracket.
      4. Slide each rear plate out of the eyelets and set them aside.
      5. Remove the two screws from each side bracket at the front of the posts.
      6. Lift the server from the rack and set it aside on an antistatic mat.
3. Attach an anti-static wrist strap.

3. Remove the Front Air Filter, power supplies,  the server Top Cover, all of the Fan Modules, the Fan Tray, rotate the Disk Device Cage.

1. On the front air filter pinch the right and left release levers inwards, pivot the air filter forward and lift it out and away from the chassis.
2. For each power supply, press the release tab to the left and pull on the handle.
3. Continue to pull on the handle to slide the power supply out of the chassis.  Set each power supply aside.
   
4. Fully loosen the two captive screws at the rear of the top cover.
5. Slide the top cover backward about .50 inch (13mm), lift top cover straight up and off from chassis.   
   Note: If the top cover will not slide back, you may have to loosen or remove the top two screws on both the left and right rack support backet.  These top screws cause the support backet to press against the top cover.
     
   
6. Remove the Fan Modules from the fan tray by pulling up on the green handle.
7. Unlock the Fan Tray from the system
   
8. Disconnect the cables attached to the disk backplane and the cable attached to the USB board.
   
9. Fully loosen the four captive screws securing the drive cage to the chassis (use a No. 2 Phillips screwdriver).
10. Rotate the drive cage to the vertical position.

4. Disconnect the  RAID HBA super capacitor cable, the air baffle, the PSU air duct and PCIe cards.

1. Grasp both ends of the HBA super capacitor cable connector, press in on the release latch, and then unplug the connector.
   
2. Move all cables away from the top of the air baffle.
   
3. Fully loosen the thumbscrew on the left bracket for the air baffle. 
   
4. Slide the left (L) thumbscrew bracket forward and off of the mounting pins and set the thumbscrew bracket aside.
5. Lift the air duct straight up and off the alignment pins.  Set the air duct aside.
6. Lift the PSU air duct up and out of the chassis. Set aside the PSU duct.
7. Disconnect cables from each of the installed PCIe cards, make sure you are aware of which cables are connected to each card and the ports to which they are attached.
8. Swing the retainers to their fully open position and lift the PCIe cards out of each socket.
9. Place each of the PCIe cards on an antistatic mat.
10. Remove each of the cables from the MB, noting their routing path and connections.
    

5. Remove the motherboard (MB)

1. Disconnect the bus cable between the PDB and the MB at the MB.
2. Disconnect all remaining cables from the MB and move them out of the way.
3. Remove the four screws securing the PDB bus bars to the MB.
4. Fully loosen the two captive screws at the front of the MB.
5. Grasp the two MB handles and slide the MB forward off of the alignment pins.
6. Tilt the MB left and slowly lift the MB up and out of the chassis.
7. Set the MB aside on an antistatic mat.

6. Replace the PDB

1. Disconnect the cable from the original PDB.
2. Loosen the two top captive screws securing the PDB to the chassis.
3. Lift the PDB bus bars out from under the motherboard and remove the PDB from the chassis.  Set this PDB aside.
4. Position the new PDB to where it installs into the chassis.  The bus bars are forward and down.
5. Place the bus bars into the channels under the motherboard, and align the PDB with two posts for the captive screws.
6. Reconnect the cable to the PDB.

7. Install the motherboard back into the chassis

1. Position the MB for installation into the chassis.  The heatsinks are toward the front of the chassis, and the ports are at the rear.
2. Lift the MB by the handles and slowly move the MB rearward until the connectors slightly protrude through the rear of the chassis.
3. Lower the MB onto the bottom alignment pins.
4. Slowly slide the MB until it drops, engaging the bottom alignment pins.
5. Continue to slide the MB forward until it seats.  The rear of the MB must be flush with the rear of the chassis. Additionally, the four holes in the MB must align with the four holes in the bus bars.
6. Tighten the two captive screws near the disk tray.
7. Install and tighten the four bus bar screws associated with the PDB.
8. Reconnect the bus cable between the PDB and the MB.
9. Reconnect the remaining cables you previously disconnected.

8. Reinstall the the remaining components, cables and air baffles:

1. Reinstall the PCIe cards
   
   1. Swing the retainers to their fully open position and insert each PCIe card back to the original socket.    
      The NVME switch card is installed in slot 5 only, the Oracle Storage 12 Gbs PCIe RAID HBA card is installed in slot 6.
        
      
   2. Swing the retainers to its fully closed position.
   3. Connect and route the cables to each of the PCIe cards as needed.
   4. Plug each cable into its connector until you hear an audible click.
2. Reinstall PSU air duct, lower the PSU duct down over the PDB and the bus bars.
   
3. Reinstall the main air duct.
   1. Move any loose cables to the gap between the DIMMs and the chassis walls.
   2. Align the air duct to where it installs into the chassis.
   3. Lower the air duct onto the alignment pins.
   4. Set the left (L) thumbscrew bracket over the alignment pins and slide it rearward. Tighten the thumbscrew.
      
4. Reinstall the Fan Tray
   
   1. Align the fan tray to where it installs into the chassis.  Two arrows on the top of power supply bay show where the fan tray aligns.
   2. Slowly lower the fan tray into the chassis.  Verify that the fan tray is properly seated and the top surfaces of the fan tray are flush with the chassis.
   3. Close the two fan tray locks.
5. Reinstall the fan modules.
   1. Align the fan module to the location where it installs into the fan tray.  The green band is on the top of the fan module and the arrow points to the rear of the chassis.
   2. Lower the fan module into the fan tray.
   3. Press the fan module down until it clicks securely in the fan tray.
   4. Repeat steps 1 through 3 for the remaining fan modules.
6. Lower the drive cage to the horizontal position.  Tighten the four screws securing the drive cage to the chassis.
7. Connect the cables to the drive backplane, and the cable attached to the USB board.  Plug each cable into its connector until you hear an audible click.
8. Complete the routing of each cable into the appropriate cable clamps.
9. Reonnect the super capacitor's cable.
10. Reinstall each power supply, align each power supply to the location where it installs into the chassis. The PDB connector is toward the chassis, and the release tab is on the left.
11. Slide the power supply into the chassis until the power supply seats and the release tab clicks.
12. Reinstall the air filter
    1. Set the bottom edge of the filter into the lip at the front of the chassis.  The left and right release levers are facing forward.
       
    2. Pivot the filter to vertical.  The release levers snap the filter into place.
13. Install the top cover
    1. Position the top cover on the chassis, with the front edge of the top cover 0.5 in. (13 mm) rearward of the front edge of the chassis.
    2. Slide the top cover forward into place.
    3. Secure the top cover with the two captive screws at the rear.
14. Reinstall or tighten the side bracket support screws.
    

9. Return the Server to operation.

1. Remove any anti-static measures that were used.
2. Return the server to it's normal operating position within the rack.
3. Re-install the power cords and any data cables that were removed.
4. Power on server. Verify that the Power/OK indicator led lights steady on.

10. Verify the system serial number/fruid data if needed.

1. The PDB is not the primary fruid container in this server so when it is replaced you will normally will not need to fix the serial number information.
2. Login to the ILOM as root and then enter the restricted shell to check the fruid values. Follow the example at the beginning of these instructions to enter restricted shell and use the showpsnc command.
3. When the PDB is replaced the Primary fruid container will likely match the Backup entries. If it does not you must enter escalation or service mode to fix it (if all three entries match this step is done).
4. Contact the TSC to request an escalation password (service mode will work also if just the copypsnc command ends up needing to be used, if the setpsnc command is needed escalation mode is required. setpsnc is not covered in this procedure).
5. Provide your TSC contact the output from the following ILOM commands- "version", "show /SYS product_serial_number", and "show /SP/clock". If the product_serial_number information does not give good output then provide the showpsnc output that was seen in step b above as well.
6. The TSC will provide an escalation password that is made up of 32 short words. Follow the example below to create a new user with the 'Service' role assigned. The Service role is required to access service or escalation modes. In the following example we will create a user named 'escuser' with the service role.
   -> cd /SP/users
   /SP/users
   -> create escuser
   Creating user...
   Enter new password: ********
   Enter new password again: ********
   Created /SP/users/escuser
   -> set escuser role=aucros
   Set 'role' to 'aucros'
   -> show escuser
   /SP/users/escuser
   Targets:
   ssh
   Properties:
   role = aucros
   password = *****
7. Set the check_physical_presence to false and then exit from the ILOM so that you can login as the newly created user.
   -> set /SP check_physical_presence=false
   Set 'check_physical_presence' to 'false'
   -> show /SP check_physical_presence
   /SP
   Properties:
   check_physical_presence = false
   
   -> exit
8. Login using the escuser login and enter escalation mode using the password that was provided by the TSC.
   ORACLESP-NX5-2 login: escuser
   Password:
   
   Oracle(R) Integrated Lights Out Manager
   
   Version 3.2.4.36 r95733
   
   Copyright (c) 2014, Oracle and/or its affiliates. All rights reserved.
   
   Warning: The system appears to be in manufacturing test mode.
   Contact Service immediately.
   
   Hostname: NSX5-2
   
   -> cd /SP/users/ecsuser/escalation
   -> set SESSION mode=escalation                            
   Password:**** **** **** **** **** *** *** **** **** **** **** **** **** **** **** **** *** *** **** *** **** **** **** *** **** **** *** **** *** *
   Short form password is:  NOSE HAAG MED
   
   [(escalation_mode) ORACLESP-NX5-2:~]#
9. Use the showpsnc command to confirm the current container values. Confirm that one of the backup containers has a serial number (the value on the PSN line) that matches the system serial number. The system serial number can be checked by comparing to the serial number RFID tag on the front left hand side of the server. After confirming that there is a valid fruid backup use the copypsnc command to write the good data from the primary to the backup1 container on the MB. The following example shows copying from primary to the backup1 but you could also copy from backup2 if needed.
   
   [(restricted_shell) ORACLESP-NX5-2:~]# showpsnc
   Primary: fruid:///SYS/DBP
   Backup 1: fruid:///SYS/MB
   Backup 2: fruid:///SYS/PDB
   
   Element           | Primary           | Backup1           | Backup2           
   ------------------+-------------------+-------------------+-------------------
   PPN                 NSX5-2              NSX5-2              NSX5-2      
   PSN                 1450NMB006          1450NMB006          0000000000
   Product Name        NETRA SERVER X5-2   NETRA SERVER X5-2   NETRA SERVER X5-2 
   
   [(escalation_mode) ORACLESP-NX5-2:~]# copypsnc Primary Backup1
   [(escalation_mode) ORACLESP-NX5-2:~]# showpsnc
   Primary: fruid:///SYS/DBP
   Backup 1: fruid:///SYS/MB
   Backup 2: fruid:///SYS/PDB
   
   Element           | Primary           | Backup1           | Backup2           
   ------------------+-------------------+-------------------+-------------------
   PPN                 NSX5-2              NSX5-2              NSX5-2      
   PSN                 1450NMB006          1450NMB006          1450NMB006         
   Product Name        NETRA SERVER X5-2   NETRA SERVER X5-2   NETRA SERVER X5-2 
   [(escalation_mode) ORACLESP-NX5-2:~]# exit
   
   
10. At this point if all of the fruid containers match and have the correct serial number data this step is done. If more than one of the fruid containers had non-valid entries then the copypsnc command should be used to copy over the valid data to the other container that is not valid. (ie. "copypsnc Primary Backup2" to copy primary to backup2) After confirming all fruid data is correct reset the ILOM to confirm that the fruid data persists through a reboot and remove the escalation user if needed.
    
    -> reset /SP
    Are you sure you want to reset /SP (y/n)? y
    Performing reset on /SP
    ..........
    
    ***login as the root user again and check the fruid data***
    
    -> set SESSION mode=restricted
    
    WARNING: The "Restricted Shell" account is provided solely
    to allow Services to perform diagnostic tasks.
    
    [(restricted_shell) ORACLESP-NX5-2:~]# showpsnc
    Primary: fruid:///SYS/DBP
    Backup 1: fruid:///SYS/MB
    Backup 2: fruid:///SYS/PDB
    
    Element           | Primary           | Backup1           | Backup2           
    ------------------+-------------------+-------------------+-------------------
    PPN                 NSX5-2              NSX5-2              NSX5-2      
    PSN                 1450NMB006          1450NMB006          1450NMB006         
    Product Name        NETRA SERVER X5-2   NETRA SERVER X5-2   NETRA SERVER X5-2 
    [(restricted_shell) ORACLESP-NX5-2]# exit
    
    -> cd /SP/users
    /SP/users
    -> delete escuser
    Are you sure you want to delete /SP/users/escuser (y/n)? y
    Deleted /SP/users/escuser
11. If trouble is encountered during any of the steps of accessing escalation mode and fixing the fruid containers please contact the TSC for assistance.

    12.  Check ILOM event log to see if any new errors are reported.

OBTAIN CUSTOMER ACCEPTANCE

WHAT ACTION DOES THE CUSTOMER NEED TO TAKE TO RETURN THE SYSTEM TO AN OPERATIONAL STATE:

Boot up system and verify full functionality.

REFERENCE INFORMATION:

Netra Server X5-2 Documentation
http://docs.oracle.com/cd/E53596_01/index.html

Oracle Integrated Lights Out Manager (ILOM) 3.2 Documentation
http://docs.oracle.com/cd/E37444_01/index.html

This solution has no attachment