Replacement procedure for How to Replace a Power Distribution Unit on an Engineered System using Sun Rack II

Asset ID:	1-71-1423708.1
Update Date:	2017-07-26
Keywords:

Solution Type Technical Instruction Sure

Solution 1423708.1 : Replacement procedure for How to Replace a Power Distribution Unit on an Engineered System using Sun Rack II

Applies to:

Exalogic Elastic Cloud X4-2 Hardware - Version X4 and later
Exalogic Elastic Cloud X3-2 Eighth Rack - Version X3 and later
Sun ZFS Backup Appliance - Version All Versions and later
SPARC SuperCluster T4-4 Full Rack - Version All Versions and later
Oracle Exalogic Elastic Cloud X2-2 Half Rack - Version X2 and later
Information in this document applies to any platform.

Goal

Replace a failed Rack PDU with a new one.

Solution

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY?

The PDU can be replaced hot, without taking power down to the rack. The 2nd operating PDU will maintain power to all of the units using the redundant power supplies in each server and switch. The only exception is on Exadata X2-2 and BDA racks with the Keyboard/Monitor tray (a non-critical item) that is powered only on the PDU-B side.

There are 2 potential risks to the operating rack:
1. From unlatching IB cables in the rack inadvertently. A loss of service in IB may result in nodes being evicted from the cluster which may crash any applications running on the rack. Care should be made if having to handle the IB cables, but the risk should be limited as the IB cables should be latched securely and should not unlatch from either switch or server IB HCA ports even with significant tension being placed on them from pulling.

2. If the rack is in a raised-floor environment, then it may be necessary to move the rack forward or backward in order to align the PDU plug with the floor tile cutout and rack floor cutout. Cut-outs should have been made large enough per the documentation, however that is not always followed particularly in older data centers. Extra care should be taken to ensure there is sufficient slack for any customer added network cables and multi-racking InfiniBand cables if present. In multi-racked environments, it is highly recommended to disable subnet manager on this rack prior to moving it, to ensure other racks are not impacted inadvertently to unplanned subnet manager failovers. In over-head cabling environments, the top cable pass through panel can be removed if necessary to facilitate passing the outlet plug through it, by undoing the screws that attach it on the top of the rack.

ADDITIONAL INFORMATION TO CONSIDER

Allow time to unpack and re-pack the PDU FRU's in the box. Make a note of how the PDU power cord(s) are coiled in the packaging, so the failed unit can be repackaged the same way.
Bring your own screwdriver with a Torx T30 and T25 bits, preferably a cordless drill or ratcheting wrench style that the handle can go out to the side so its parallel to the PDU for more maneuverability/hand access.

There are basic T25 and T30 hand wrenches in the rack ship kit if necessary, but they are not easy tools to work with.

Removal with side-panel off takes about 5 minutes to remove and 10-15 minutes to install. This is faster, but not necessary to the replacement.
If side-panel access is not possible, it takes about 10 minutes to remove, and about 25-30 minutes to install typically.

These times were using a cordless drill with a Torx T30 bit and regular T25 screw driver as filmed. Maintenance space and quality of the tools used may or may not make this longer.
Allow about 30 minutes longer if you need to use the hand wrench tool that is provided with the rack.
The majority of the installation time is spent re-attaching the server power cords to the designated outlet.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

Removal:

1. If the PDU monitor is not the reason for replacing the PDU, then use its display to identify and record all its network settings. To check the current Network address, hold down the red button until it starts counting down 5 4 3 2 1 0. While it is counting down, release the red button and then press it again once. This will reset the monitor and you will see it flash and turn off and then display the network details, firmware version etc as it boots. Make a note of these values for later configuring the new PDU.

2. Disconnect the PDU monitor network cable from the PDU.

3. Turn off all the breakers for each inlet group on the PDU being replaced.

4. Unplug the PDU power cord(s) from the data center power outlets. Keep the PDU power cord plugs in a location out of the way. If the PDU power cord(s) are using overhead routing, then re-locate them so the plug ends are somewhere they will not fall or hit anybody before starting on the PDU itself.

5. Disconnect the rack Ground cable from the top of the PDU.

6. If the replacement is on the PDU-A side (left when looking from the rear), then skip this step. If the replacement is on the PDU-B side (right when looking from the rear) and any square cable arms for the rack are being used for IB cables (either for routing multi-rack IB cables or for routing in-rack IB cable slack where harnesses are not installed), then you will need to also undo the T25 screws holding the square cable arms to the rack and pull the cable arm (with IB cables within) into the middle space in the rack, out of the way of the PDU exit/entrance path.

Tips:
- Do not attempt to un-velcro any of the cables attached to those arms. If you do need to un-velcro anything, be careful to tie it back down the same way.

- This step is done before unplugging the server power cords so you have extra hand access between the IB cables and the rack side wall.

- Be careful not to put too much tension on the square cable arms as they can be easily broken.

- There should be sufficient slack in any IB cables routing through the arms to allow them to move far enough to be out of the way. The IB cables should be latched securely and should not unlatch from the switch or server IB HCA ports even with significant tension being placed on them.

- Cable arm mount screw holes are spring loaded and may fall down the rack groove they sit in. If this occurs, slide them up to where they were originally when re-attaching them.

7. Unplug all the server power cables going into the PDU before unscrewing them. Try to keep them bunched in group bundles near the outlets they were removed from. There should be plenty of access room.

8. Unscrew the PDU from the rack frame by removing the T30 screws from top and bottom of the PDU.

Tips:

- There are two larger T30 head screws that attach a type of L-bracket that attaches to the rack, there is a T25 smaller screw(s) and nut that attaches the PDU to the L-bracket. Do NOT remove the T25 screws or the nuts that hold in the L-brackets until it is out of the rack.

- The L bracket has slots to allow it to slide back and forth before screwing into the rack mount holes. Note the locations of the holes where the screws are attached, so the replacement PDU can sit at the same location. Typically it is located such that the PDU sits about 1" (25mm) back from the rear rack frame to allow access to the breaker switches.

9. Angle and maneuver the PDU out of the rack vertically, sliding it out between the rack frame and any server and switch cable arms and cabling.

10. Hold up the PDU or lay it down if there is room while maneuvering the PDU power cord(s) through the rack. If necessary, cut the cable ties (or velcro straps) that hold the PDU power cord(s) flush with the bottom side of the PDU. This will give you more flexibility to wiggle the rest of it through. Pull the PDU power cord(s) as near to the bottom or top of the rack as possible where there is more room between servers to get the outlet plug through its routing hole.

Installation:

1. The L-brackets used to attach the PDU to the rack does not come with the PDU FRU, and need removed from the old PDU and re-attached to the new PDU.
a) Remove the smaller T25 Torx screws, and loosen the nut slightly on top and bottom to get the PDU loosened from the L-brackets. The nut only needs to be loosened, not removed, as the screw it holds is sitting in a groove. Slide the L-bracket off the old PDU.
b) Slot the L-bracket onto the new PDU the same way, tightening the nuts and re-installing the original T25 Torx screws.
Repeat for L-brackets used on top and bottom of the PDU.

2. Route the PDU power cord(s) through the rack and out of the way to near where the data center outlets are.

Tips:

- If space allows, the PDU should be laid down next to the rack while doing this, to prevent it falling over.

- The PDU power cord(s) can be optionally tied top or bottom as needed if you have side-panel access and more cable ties, or if it is routed as much as possible before starting the install, however it is preferable if you do not cable tie the PDU power cord(s) onto the new PDU for added maneuverability during install.

- If the PDU power cord(s) are using overhead routing, then re-locate them so the plug ends are somewhere they will not fall or hit anybody while the PDU is being installed into the rack.

3. Slide the new PDU into the rack vertically by angling and maneuvering it in until the L-brackets sit on the top and bottom cross members of the rack frame.

4. Screw the PDU into the rack frame using the T30 Torx screw(s) on the top and bottom. Line up the same mounting holes and slots that were used to hold it in, so that it sits ~1" (25mm) back from the rear rack frame for breaker access. Look at the other PDU if unsure of the positioning relative to the rack frame.

5. Re-connect the server and switch power cords to the PDU outlets using the labels on the power cords as a guide where "G5-0" would indicate PDU Group 5 Outlet 0 as labeled on the PDU.

6. Re-connect the rack Ground cable at the top of the PDU.

7. If applicable, re-attach any square cable arm brackets to the rack frame using the T25 Torx screw(s) that were holding the arms on originally. Note the cable arm mount screw holes are spring loaded and may fall down the rack groove they sit in. If this occurs, slide them up to where they were originally when re-attaching them. The cables routing in the arms should be able to stay within the arms. Be careful not to put too much tension on the square cable arms while attaching them, as they can be easily broken.

8. Re-connect the PDU power cord(s) to the data center outlets.

9. Turn on all the breakers for each inlet group on the PDU.

10. Re-configure the PDU network monitor to the customer's network settings. To check the current Network address hold down the red button until it starts counting down 5 4 3 2 1 0. While it is counting down, release the red button and then press it again once. This will reset the monitor and you will see it flash and turn off and then display the network details, firmware version etc as it boots. Use this network information to get your laptop to access it via browser. It may also be necessary to update the firmware. Follow the instructions in the Exadata X3 EIS checklist and update with the customer's network information.

11. Re-connect the PDU network monitor to the customer's network switch or the rack internal management switch, where applicable.

The following may be used for reference and training:

Video - PDU Removal (Length - 5:18, Size 37MB) Removal steps are viewable in the attached video.

Video - PDU Installation (Length - 5:35, Size 38MB) Install steps are viewable in the attached video

Video - Short version (Length - 2:21, Size 36MB) Shorter version video of PDU removal and installation. Also available for streaming on OTube.

VERIFICATION ACTIONS FOR THE ENGINEER TO COMPLETE IN ORDER TO RETURN THE SYSTEM TO THE CUSTOMER FOR ACCEPTANCE

Verify the PDU is functioning, and all servers and switches are no longer showing failed for the power supplies. On servers, use the front LED's. On the IB and Cisco switches, look through the vented filler panels to ensure both sides PSU's are green. On the KVM switch on the left side ensure both power LED's for A and B are green. If PDU-B was replaced, verify the Keyboard/Monitor tray comes on if applicable.

On the PDU, verify the current monitor is displaying the current draw on each input and outlet group. Use the button to toggle the direction and input being displayed (on units with more than 1 3-phase input).

From a workstation or laptop on the customer's network, verify the PDU can be accessed over the network to display the current monitor in a web browser. The PDU should be reconfigured for the customer's network, per the detailed instructions in the EIS checklist for the appropriate system. The PDU firmware may need to be updated per the EIS checklist, also referencing Note 1543240.1 for instructions.

References

<NOTE:1543240.1> - How to Update Firmware for the PDU in Your Engineered System
<NOTE:1377637.1> - How to replace a Power Distribution Unit (PDU) on an Engineered System using Sun Rack II
<NOTE:778.1> - Troubleshooting Video Issues in MOS

Attachments

This solution has no attachment