Mx-32 - How to replace a Faulty Midplane

Asset ID:	1-71-1547512.1
Update Date:	2016-01-20
Keywords:

Solution Type Technical Instruction Sure

Solution 1547512.1 : Mx-32 - How to replace a Faulty Midplane

Applies to:

SPARC M6-32 - Version All Versions and later
SPARC M5-32 - Version All Versions and later
Oracle SuperCluster M6-32 Hardware - Version All Versions and later
Information in this document applies to any platform.

Goal

CAP PROBLEM OVERVIEW: Mx-32 - How to Replace a Faulty Midplane

To report errors or request improvements on this procedure, please go to My Oracle Support, and put a comment on Doc ID: 1547512.1

This Canned Action Plan describes all necessary steps to replace a Faulty Midplane, which are described in great detail in the the SPARC M5-32 Server Service Manual, Servicing the Midplane.

Replacing a Faulty midplane is such a complex operation, you do need to read, understand and use the SPARC M5-32 Server Service Manual, Servicing the Midplane section from the service manual, the illustrations around the actual Faulty Midplane replacement are complimentary to this written action plan.

ESD Caution:

Circuit boards and drives contain electronic components that are extremely sensitive to static electricity. Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these boards. Do not touch the components along their connector edges.
Use a Antistatic Wrist strap. Attach one end of the strap to your wrist and the other end to the chassis, depending on what type of strap you use, with the adhesive end or the metal plug.
Use an Antistatic Mat. Place ESD-sensitive components such as motherboards, memory, and other PCBs on an antistatic mat.

Contamination Caution:

Dust particles of packaging material are number one cause of datacenter contamination. Make sure to remove all packaging material, up to the ESD safe packaging material, while still being outside the datacenter.

Solution

DISPATCH INSTRUCTIONS

WHAT SKILLS DOES THE ENGINEER NEED:(IS A SITE ENGINEER AVAILABLE?)

Mx-32 Product Training/Experience

TASK COMPLEXITY: 4

TIME ESTIMATE: 300 minutes COLD replacement

FIELD ENGINEER INSTRUCTIONS

WHAT STATE SHOULD THE SYSTEM BE IN TO BE READY TO PERFORM THE RESOLUTION ACTIVITY? :

Physical Domains must be shut down. Server must be powered off.

WHAT ACTION DOES THE ENGINEER NEED TO TAKE:

Caution:

The midplane weighs 80 lbs (36 kg). Two Oracle engineers are required to move it.
Do not remove the protective plastic covers from the connectors on the midplane until you are ready to plug a cable connector into a socket
Open the shipping package carefully. It will be reused to return the faulty midplane to Oracle
To replace the scalability assembly, prepare ESD safe storage space for 36 fans, 12 Scalability Switch Modules, 2 Service Processors, 2 Clock Boards, 2 Fan Cages, 1 Front Filler panel, 1 Scalability card cage, 1 Scalability assembly, 16 CPU/Memory boards and 2 Midplanes
Record the slot number for each component, and stack them in order. You must place them back in their original order later

1. Unpack and inspect the replacement midplane FRU, it should contain:

midplane, with protective connector covers
torque wrench
extra protective plastic covers for cable connectors
cable ties
two midplane handles
all tools, covers, handles and unused cable ties are to be returned to Oracle

2. Halt the server and turn off all AC power:

Turn the key switch on the Front Panel of the server to the Service position.
Gracefully shutdown the server by running the ILOM command "stop /System" (requires Oracle ILOM Reset and Host Control (r) user role).
Turn off all external breakers for the server.
For maximum safety, disconnect all six power cords from their respective AC outlets.

3. Verify that it is safe to disconnect the power cords:

All six external breakers for the server must be in the Off position.
All six power cords for this server must be unplugged from their respective AC outlets.
No LEDs on the server should be lighted.
You should not hear any fans running in the server.

4. Remove the front filler panel. check M5-32 How to Service the Front Filler Panel Assembly [ID 1539427.1] for details

5. Remove all 16 Fan modules from the Upper Fan Cage and Remove all 16 Fan modules from the Lower Fan Cage, check M5-32 How to replace a Faulty Fan Module [ID 1530231.1] for details

6. Remove both Service Processors from the Scalability Cage, check M5-32 - How to Replace a Faulty Service Processor (SP) [ID 1529334.1] for details

7. Remove both Clock Boards from the Scalability Cage, check M5-32 - How to Replace a Faulty Clock Board [ID 1528995.1] for details

8. Remove all 12 Scalability Switch Boards from the Scalability Cage, check M5-32 - How to Replace a Faulty Scalability Switch Board (SSB) [ID 1529636.1] for details

9. Remove all 4 Fan modules from the Scalability Cage, check M5-32 How to replace a Faulty Fan Module [ID 1530231.1] for details

10. Remove the upper fan cage. check M5-32 - How to Replace a Faulty Upper Fan Cage Assembly [ID 1531620.1] for details

11. Remove the lower fan cage. check M5-32 - How to Replace a Faulty Lower Fan Cage Assembly [ID 1531637.1] for details

12. Remove the Scalability Card Cage, check M5-32 How to Replace a Faulty Scalability Card Cage [ID 1546735.1] for details

13. Remove the Scalability Assembly, check M5-32 How to replace a Faulty Scalability Assembly [ID 1538201.1] for details

14. Remove all 16 CPU Memory Unit (CMU) or CPU memory unit (CMU) filler panel from the rear from the server:

for CPU Memory Unit (CMU) removal, check M5-32 How to Replace a Faulty CPU Memory Unit (CMU) [ID 1529502.1] for details
for CPU Memory Unit (CMU) filler panel removal, check M5-32 How to Replace a Faulty CPU memory board (CMU) filler panel [ID 1529544.1] for details

Complexity Caution:

Replacing a Faulty midplane is such a complex operation, you do need to read, understand and use the SPARC M5-32 Server Service Manual, Servicing the Midplane section from the service manual, the illustrations around the actual Faulty Midplane replacement are complimentary to this written action plan.

15. At the front of the server, insert cable ties in slots on the bottom of the power system cage:

the cables in front of the midplane will be raised up and tied in place so that the midplane can be pulled out through the front of the server
you will need to clear a path for the midplane to slide forward

Note – Do not place screws or other loose hardware on the “shelves” of the midplane. Small items can easily fall inside the cabinet

16. Unlock and detach a CMU data cable assembly:

there are two metal locking levers on each CMU data cable assembly. The metal locking levers push against plastic locking tabs that are part of the CMU power cable assembly
push up slightly on the plastic locking tab, then pull down a metal lever
repeat for the second lever
pull the CMU data cable assembly out of the midplane and let the assembly hang loose while you remove other assemblies in steps
repeat the above 3 steps until all the data cable assemblies are detached from the midplane

17. Unlock and detach the CMU power cables:

remove two T20 Torx screws that hold the CMU power cable assembly to the midplane, then pull the cable assembly out of the midplane
take care not to drop the screws. These are not captive screws

18. Use a T20 Torx driver to detach the SPP cables

19. Raise the CMU data, power, and SSP cable assemblies and tie them as neatly as possible to the top of the midplane area:

keep the cables arranged as neatly as possible, or they will become unmanageable
above the midplane area, there are four bundles of CPU power cables, four bundles of CPU data cables, and four bundles of IO cables

Note – Take care not to drop the screws. These are not captive screws

20. Use a T20 Torx screwdriver to detach the scalability assembly cables from the sides of the cabinet:

tape the upper scalability assembly brackets as high as possible to the side of the cabinet
place the lower scalability assembly brackets on the floor of the cabinet

21. In the shipping carton for the replacement midplane, locate two detachable midplane handles. They are marked “Left” and “Right”"

22. Attach the handles to the slides:

extend a slide fully
insert the hooked tab on the handle into the slide first, then lower the rear of the handle so the remaining two tabs drop into the slide
push the slide back into the chassis until the handle aligns with the side of the midplane
insert the keyed pins on the handle into the keyholes on the side of the midplane
raise the spring-loaded lock pin, then align the pin with the corresponding hole in the base of the midplane
install the second handle in the same manner

22. At the rear of the server, remove four screws that are located inside the CMU cage area

23. At the midplane, loosen 13 mounting screws

24. Holding the handles, pull the midplane out until the slides are fully extended

25. Use the handles to lift the faulty midplane off the slides and out of the server

26. Temporarily set the faulty midplane aside on a ESD safe surface. In a later step, the midplane will be packed in the shipping carton that contains the replacement midplane

27. Remove the handles to use with the replacement midplane.

28. Stand the replacement midplane vertically on a static safe work space.

29. Attach the handles to the sides of the replacement midplane:

insert the keyed pins on the handle into the keyholes on the side of the midplane
raise the spring-loaded lock pin, then align the pin with the corresponding hole in the base of the midplane
install the second handle in the same manner

30. Place the midplane on the slides:

extend both slides fully
use the midplane handles to carry the midplane to the slides
tilt the top of the midplane slightly away from server to insert the hooked tabs on the handles into the near ends of the slides, then lower the midplane so the rear tabs on the handles drop into the slides

31. Remove the plastic covers from the connectors on the midplane. Do not discard the plastic covers. The covers will be returned to Oracle !!

Caution – Because of the weight of the midplane, it is easy to accidentally damage the connectors. Push the midplane slowly until the guide pins and the connectors are engaged

32. Push evenly on both handles to slowly slide the midplane into the server

33. Loosely install 13 captive T20 Torx screws on the front of the midplane. Do not tighten the screws yet

34. At the rear of the server:

install the four T20 Torx screws through the CMU cage to the rear of the midplane
tighten these screws to 15 in-lb

35. At the front of the server tighten the midplane screws:

top row sequence number from left to right is 9-7-5-3-1-2-4-6-8
start in the middle, then alternate right and left until all nine screws are tightened to 15 in-lb
tighten the remaining four screws in any order to to 15 in-lb

36. Reattach cables to the midplane:

install the I/O power cables, check M5-32 How to replace a Faulty Cabled Lower Bus Bar Assembly [ID 1547807.1] for details
install the I/O data cables, check M5-32 How to Replace a Faulty IO Data Cable [ID 1547733.1] for details

37. Reattach the connectors and brackets to the sides of the cabinet

38. Install all 16 CPU Memory Unit (CMU) or CPU memory unit (CMU) filler panel from the rear from the server:

for CPU Memory Unit (CMU) removal, check M5-32 How to Replace a Faulty CPU Memory Unit (CMU) [ID 1529502.1] for details
for CPU Memory Unit (CMU) filler panel removal, check M5-32 How to Replace a Faulty CPU memory board (CMU) filler panel [ID 1529544.1] for details

39. Install the Scalability Assembly, check M5-32 How to replace a Faulty Scalability Assembly [ID 1538201.1] for details

40. Install the Scalability Card Cage, check M5-32 How to Replace a Faulty Scalability Card Cage [ID 1546735.1] for details

41. Install the lower fan cage. check M5-32 - How to Replace a Faulty Lower Fan Cage Assembly [ID 1531637.1] for details

42. Install the upper fan cage. check M5-32 - How to Replace a Faulty Upper Fan Cage Assembly [ID 1531620.1] for details

43. Install all 4 Fan modules from the Scalability Cage, check M5-32 How to replace a Faulty Fan Module [ID 1530231.1] for details

44. Install all 12 Scalability Switch Boards from the Scalability Cage, check M5-32 - How to Replace a Faulty Scalability Switch Board (SSB) [ID 1529636.1] for details

45. Install both Clock Boards from the Scalability Cage, check M5-32 - How to Replace a Faulty Clock Board [ID 1528995.1] for details

46. Install both Service Processors from the Scalability Cage, check M5-32 - How to Replace a Faulty Service Processor (SP) [ID 1529334.1] for details

47. Install all 16 Fan modules from the Upper Fan Cage and Install all 16 Fan modules from the Lower Fan Cage, check M5-32 How to replace a Faulty Fan Module [ID 1530231.1] for details

48. Install the front filler panel. check M5-32 How to Service the Front Filler Panel Assembly [ID 1539427.1] for details

49. Verify all six external breakers are still in the Off position.

50. Warn bystanders that you are preparing to restart the server.

51. Turn on all AC power and start the server:

Connect all six AC cords to their respective AC outlets.
Turn on all external breakers for the server.
Turn the key switch on the Front Panel of the server to the On position.
Start the server by running the ILOM command "start /System" (requires Oracle ILOM Reset and Host Control (r) user role).

52. Prepare the failed Midplane outside the datacenter for return to Oracle. Also return any extra covers that were included with the replacement component.

Attachments

This solution has no attachment