| Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback |
| Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition | ||
|
|
||
 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Solution Type Technical Instruction Sure Solution 1992940.1 : Sun Storage 7000 Unified Storage System: PARTNER SERVICE - How To Replace A Motherboard In A 7x10 NAS Appliance
In this Document
Oracle Confidential PARTNER - Available to partners (SUN). Applies to:Sun Storage 7210 Unified Storage System - Version All Versions to All Versions [Release All Releases]Sun Storage 7310 Unified Storage System - Version All Versions to All Versions [Release All Releases] Sun Storage 7410 Unified Storage System - Version All Versions to All Versions [Release All Releases] Sun Storage 7110 Unified Storage System - Version All Versions to All Versions [Release All Releases] 7000 Appliance OS (Fishworks) GoalThis document provides the steps that Partner engineers need to take for replacing the motherboard for Unified Storage 7x10 series.
This procedures is applicable to the following products. 7110, 7210, 7310, 7410 and 7310C/7410C (Cluster Models)
Partner engineers who are trained on the x4240, x4540, x4140 and x4440 platforms. SolutionThis document outlines the steps to replace a motherboard in a Sun Storage, Unified Open Storage 7x10 Series (7110, 7210, 7310, 7410). After completing the following procedure, the system will have a replacement motherboard, will be running with the approved 7x10 series appliance BIOS and have the correct BIOS settings. NOTE #1: The onboard network interfaces will now have new MAC addresses. Target AudiencePartner engineers who are trained on the x4240, x4540, x4140 and x4440 platforms. You will need
This procedure will require you to access the SP via a serial connection and across the network using a browser. Both of these steps could be done using customer equipment if they have the necessary kit setup, however don't assume this and bring along a laptop with appropriate cables. OverviewIf you are familiar with motherboard swaps in x4240, x4540, x4140 or x4440 platforms, the physical swapping of the motherboard is identical. Full details on the official process can be found in the appropriate service manual: (to access the SSH 7x10 listings, login to MOS)
What is special here is a specific SP firmware and BIOS, coupled with a particular BIOS configuration.
If you are swapping a motherboard in a 7310/7410 cluster, then there is the possibility that the newly replaced motherboard will now have a newer revision BIOS than the partner head. To address this, once the motherboard replacement is complete, repeat the steps again for the partner head starting at section Load SP Firmware/BIOS. This includes instructions for identifying the BIOS version.
Detailed Steps
If the SP is available before starting work, it is a good idea to make a note of the network configuration of the SP. This will make life easier when configuring the replacement SP; it is physically located on the motherboard therefore requires the network configuration restoring.
-> cd /SP/network
/SP/network ... pendingipaddress = 10.18.18.21 pendingipdiscovery = static pendingipgateway = 10.18.16.1 pendingipnetmask = 255.255.240.0 If the SP is unavailable, you will need to speak to the customer about getting the network configuration for the SP. You will need an IP address, subnet mask and default router/gateway.
There's 4 bits of information which we should record at this point for safe keeping. This information is known by different names by different subsystems. These are:
Out of these, it's the last one that is really important to the appliance. The objective of this part of the procedure is to make the physical FRU information for the new motherboard match that of the original board. This keeps everything consistent with the appliance and ensures that the "phone home" data continues to be recorded against the same registered details. Again, like the SP network configuration, if it is possible to access the SP before changing the motherboard, then you can get this information from the SP. The first example comes from a 7110 (underlying platform is x4240): -> cd /SYS /SYS -> show ... Properties: type = Host System chassis_name = SUN FIRE X4240 chassis_part_number = 540-7618-XX chassis_serial_number = 0226LHF-0822B400WL chassis_manufacturer = SUN MICROSYSTEMS product_name = SUN FIRE X4240 product_part_number = 602-4172-01 product_serial_number = 0825QAS047 product_version = (none) product_manufacturer = SUN MICROSYSTEMS power_state = On Here you can see all 4 of the FRU fields (Chassis Part/Serial Number, Product Part/Serial Number) populated. The next example is from a 7210 (underlying platform is x4540): Properties: type = Host System chassis_name = SUN FIRE X4540 chassis_part_number = 000-0000-00 chassis_serial_number = 0000000-0000000000 chassis_manufacturer = SUN MICROSYSTEMS product_name = SUN FIRE X4540 product_part_number = 602-3915-01 product_serial_number = 0902AMF003 product_version = (none) product_manufacturer = SUN MICROSYSTEMS power_state = On
As you can see, the Chassis Part and Serial numbers are blank. Empirical evidence shows this to be the case in all systems tested. As mentioned previously, it's the Product Serial Number that the appliance cares about, so make a note of that. All 4 bits of information can be found printed on physical labels somewhere on the original box. For the 7110, 7310 and 7410 systems, a label can be found at the front, on top of the system chassis, with the equivalent 7210 label being found on the back of the chassis. This label identifies the Product Serial Number as Sys SN, ie System Serial Number. Like I say, different names for the same things... Finally, the yellow shipping note has the Product Serial Number printed as Serial Number.
The 7110, 7310 and 7410 systems share the same motherboard. For the 7210 system, the FRU is likely to be the System Controller Assembly which consists of the motherboard and I/O board in a single unit. You should replace the whole assembly and not try to change just the motherboard. The directions for changing the motherboard can be found on the inside of the system lid. You will also need cleaning alcohol, wipes and heat sink compound. Experience says that the heat sink compound (in a syringe) is not always shipped with motherboards from logistics. Use the alcohol and wipes to remove the old heat sink compound from the top of the CPU(s) and heat sink(s) before inserting into new motherboard.
The official procedure for swapping motherboards in these systems is generic. The 7x10 series systems have specific configurations for how the HBA or SAS cables are attached to the motherboard. It is strongly recommended that careful note is taken of how the original system was cabled.
Once reconnected, the box will power up. With a serial connection into the SP (labelled SER MGT) log into the SP. Username "root" (password default is "changeme") and configure the SP networking (substitute appropriate addresses and subnet mask) and SP hostname (the SP hostname is optional and not a requirement):
-> cd /SP/network
/SP/network -> set pendingipaddress=10.18.17.215 Set 'pendingipaddress' to '10.18.17.215' -> set pendingipdiscovery=static Set 'pendingipdiscovery' to 'static' -> set pendingipgateway=10.18.16.1 Set 'pendingipgateway' to '10.18.16.1' -> set pendingipnetmask=255.255.240.0 Set 'pendingipnetmask' to '255.255.240.0' -> set commitpending=true Set 'commitpending' to 'true' -> cd .. /SP -> set hostname=v-ss7110b-sp-gmp03 Set 'hostname' to 'v-ss7110b-sp-gmp03' NOTE: THESE ADDRESSES ARE EXAMPLES. USE THE VALUES NOTED DOWN EARLIER
The SP firmware/BIOS will likely be incorrect for the 7x10 series. In the case where the replacement board has shipped with newer 3.x based SP firmware, this procedure is required so as to downgrade to a Unified Storage 7x10 series supported 2.x version. First check the SP firmware version when logging in: ss7410b-h2 login: root Password: Sun(TM) Integrated Lights Out Manager Version 3.0.6.16 r50656 Copyright 2009 Sun Microsystems, Inc. All rights reserved. This is an example of a Unified Storage 7x10 series unsupported 3.x SP firmware. Next check the BIOS with: -> cd /SYS/MB/BIOS /SYS/MB/BIOS -> show /SYS/MB/BIOS Targets: Properties: type = BIOS fru_name = SYSTEM BIOS fru_description = SYSTEM BIOS fru_manufacturer = AMERICAN MEGATRENDS fru_version = 0ABMN052 fru_part_number = AMIBIOS8
The "fru_version" here should be:
The 7110, 7310 and 7410 systems share the same system board, hence use the same BIOS. The SP firmware and BIOS are updated at the same time from a single image. It is handy to have a terminal window with a serial connection to the SP open at this stage. The updating is done via a browser, but the subsequent reboot can be observed via the open serial connection. To update, point a browser at the SP, log in as root and follow these steps: 1. Click on main Maintenance tab 2. Firmware Upgrade will be the default and correct subtab 3. Click on "Enter Upgrade Mode" 4. Confirm this action with the pop up 5. Click on "Browse" and select the appropriate image from your local filesystem 6. Click on "Upload" 7. Wait for upload to complete and the verification to succeed 8. You will now see a Summary Table of the SP firmware and BIOS versions (Existing vs New). Confirm that "Preserve existing configuration" is checked for the SP Firmware. 9. Click on "Start Upgrade" 10. Confirm this action with the pop up 11. Now wait for the upgrade to proceed. If the head was up at this point, it will be cleanly shutdown. Warning! Do not interrupt the update. Leave the browser undisturbed until the update is complete. 12. When finished, you will see "Upgrade Complete" and the SP will reboot. This is where it is handy to have the SP serial window open.
The objective here is to store the FRU identity of the original system on the newly installed components. You should have 4 bits: Chassis Part/Serial Number, and Product Part and Serial Numbers. The Product Part Number should be preprogrammed, however it is the Product Serial Number that the appliance cares about, so that's what we need to update using servicetool. To do this, logout of the SP as "root" and log back in as "sunservice" (same password):
(flash)root@v-ss7110b-sp-gmp03:~# servicetool --fru_product_serial_number
Servicetool is going to update the product serial number in mainboard and pdb FRUs. Do you want to continue (y|n)? y When entering values, do not use quotes; If you require embedded quotes, escape them with three backslashes; e.g. \\\" What is the new product serial number? 0825QAS047 The product serial number has been updated. The new serial number is: "0825QAS047" Updating FRUs... done NOTE: THE SERIAL NUMBER HERE IS AN EXAMPLE. USE THE VALUE NOTED DOWN EARLIER At this stage you can also update the Chassis Part/Serial numbers using servicetool with the flags --fru_chassis_serial_number and --fru_chassis_part_number. With all the FRU details updated, reboot the SP:
(flash)root@v-ss7110b-sp-gmp03:~# reboot
/sbin/reboot INIT: Sending processes the TERM signal ... If the servicetool reports the serial number as NULL
[(flash)root@SUNSP0003BAD87568:/]# more /tmp/product.good Check containers for 2.0 ILOM/SP for serial number Container Power Distribution Board
Save the entire motherboard FRUID image, in case it later needs to be restored to its original state: # frutool -r mb > /persist/mb.bin # frutool -t pb --read_cmm_product_area > /dev/shm/product.info View the file by issuing the following command to verify the previous step was successful: # cat /dev/shm/product.info Write the product information to the motherboard by issuing this command: # frutool -t mb --write_cmm_product_area < /dev/shm/product.info
Once rebooted, log in using "root" and confirm the FRU information is now correct: -> cd /SYS /SYS -> show ... Properties: type = Host System chassis_name = SUN FIRE X4240 chassis_part_number = 540-7618-XX chassis_serial_number = 0226LHF-0822B400WL chassis_manufacturer = SUN MICROSYSTEMS product_name = SUN FIRE X4240 product_part_number = 602-4172-01 product_serial_number = 0825QAS047 product_version = (none) product_manufacturer = SUN MICROSYSTEMS power_state = On
The SP firmware and BIOS will now have been updated to the correct 7x10 version. Now you must configure some specific BIOS settings. Boot the head and enter setup with: -> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS -> start /SP/console Are you sure you want to start /SP/console (y/n)? y Serial console started. To stop, type ESC ( Once you see the initial BIOS banner, hit CONTROL-E a few times; this will trigger the BIOS Setup menu after the initialisation. You can drop back to the SP with ESC-( NOTE: Escape, followed by shift 9 -- at least open parenthesis is usually on shift 9. If the initialisation hangs on a 7310/7410, and it is part of a cluster with the other head up and in service, disconnect the SAS cables to the J4400 JBODs, drop back to the SP and reset with: Serial console stopped. -> reset /SYS Are you sure you want to reset /SYS (y/n)? y Performing hard reset on /SYS -> start /SP/console Are you sure you want to start /SP/console (y/n)? y Serial console started. To stop, type ESC (
Once into the BIOS Setup screen, start by loading factory defaults. To do this, use the right arrow key to move over to the "Exit" menu. Down arrow to "Load Optimal Defaults" and, then again to confirm the popup asking "Load Optimal Defaults". Now follow the specific instructions for the appropriate appliance: BIOS Settings for 7110, 7210, 7310, and 7410
Disable PCIPnP Option-ROM scanning for slots 1-5 Disable I/O allocation Use the right arrow key to page over to "PCIPnP" menu. Use the down arrow to highlight: Scanning OPROM on PCI-E Slot1 Enabled Press return and select "Disabled". This will now appear as: Scanning OPROM on PCI-E Slot1 Disabled Repeat this for slots 2-5 (the last slot is off the bottom of the screen). You should now have: Scanning OPROM on PCI-E Slot0 Enabled Scanning OPROM on PCI-E Slot1 Disabled Scanning OPROM on PCI-E Slot2 Disabled Scanning OPROM on PCI-E Slot3 Disabled Scanning OPROM on PCI-E Slot4 Disabled Scanning OPROM on PCI-E Slot5 Disabled Just below these OPROM settings are a group of settings which allow IO allocation to be disabled per-slot. Disable PCI-E slots 1-4. Only slots 0 and 5 should be enabled. It should look like: IO Allocation on PCI-E Slot0 Enabled IO Allocation on PCI-E Slot1 Disabled IO Allocation on PCI-E Slot2 Disabled IO Allocation on PCI-E Slot3 Disabled IO Allocation on PCI-E Slot4 Disabled IO Allocation on PCI-E Slot5 Enabled On boot, you will see the following warning message from the BIOS: Warning: IO resource not allocated This is an expected message and does not indicate a failure. Exiting BIOS Setup Use right arrow to page over to "Exit". Press for the default "Save Changes and Exit", and again to confirm the action with the pop up.
Disable PCIPnP Option-ROM scanning for all slots Disable I/O allocation Use the right arrow key to page over to "PCIPnP" menu. Use the down arrow to highlight: Scanning OPROM on PCI-E Slot0 Enabled Press return and select "Disabled". This will now appear as: Scanning OPROM on PCI-E Slot0 Disabled Repeat this for slots 2-5 (the last slot is off the bottom of the screen). You should now have: Scanning OPROM on PCI-E Slot0 Disabled Scanning OPROM on PCI-E Slot1 Disabled Scanning OPROM on PCI-E Slot2 Disabled Just below these OPROM settings are a group of settings which allow IO allocation to be disabled per-slot. Disable PCI-E slot 0. Only slots 1 and 2 should be enabled. It should look like: IO Allocation on PCI-E Slot0 Disabled IO Allocation on PCI-E Slot1 Enabled IO Allocation on PCI-E Slot2 Enabled On boot, you will see the following warning message from the BIOS: Warning: IO resource not allocated This is an expected message and does not indicate a failure. Exiting BIOS Setup Use right arrow to page over to "Exit". Press for the default "Save Changes and Exit", and again to confirm the action with the pop up.
Disable PCIPnP Option-ROM scanning for all slots Disable I/O allocation Configure boot drives Use the right arrow key to page over to "PCIPnP" menu. Use the down arrow to highlight: Scanning OPROM on PCI-E Slot0 Enabled Press return and select "Disabled", followed by return. This will now appear as: Scanning OPROM on PCI-E Slot0 Disabled Repeat this for slots 1-2. You should now have: Scanning OPROM on PCI-E Slot0 Disabled Scanning OPROM on PCI-E Slot1 Disabled Scanning OPROM on PCI-E Slot2 Disabled Just below these OPROM settings are a group of settings which allow IO allocation to be disabled per-slot. Disable PCI-E slots 1 and 2. Only slot 0 should be enabled. It should look like: IO Allocation on PCI-E Slot0 Enabled IO Allocation on PCI-E Slot1 Disabled IO Allocation on PCI-E Slot2 Disabled Next, arrow over to the Boot menu. Select the last item: "Hard Disk Drives" and press return. The list should include only 2 drives (the 2 internal SATA drives) with labels like: SATA:11M-<drive model> SATA:12M-<drive model> If this list includes anything else (such as readzilla cache devices with a 'STEC MACH8' string, or JBOD attached drives) you'll need to remove them from the list by selecting the boot position and setting it to 'Disabled' for each of non-boot drives. If the list is full (with 16 drives) you will not be able to edit the list. However, the change to the OPROM settings above will cause the JBOD drives to disappear from the list on the next boot. You will need to exit and save changes and immediately re-enter the BIOS menu on the next boot (CTRL-E). Exiting BIOS Setup Once you've removed any readzilla cache or JBOD drive entries from the "Hard Disk Drives" list, press ESC to exit the "Hard Disk Drives" menu, then arrow right to the "Exit" menu. Press for the default "Save Changes and Exit", and return again to confirm the action with the pop up. On boot, you will see the following warning message from the BIOS: Warning: IO resource not allocated This is an expected message and does not indicate a failure.
Disable PCIPnP Option-ROM scanning for all slots Disable I/O allocation Configure boot drives Use the right arrow key to page over to "PCIPnP" menu. Use the down arrow to highlight: Scanning OPROM on PCI-E Slot0 Enabled Press return and select "Disabled", followed by return. This will now appear as: Scanning OPROM on PCI-E Slot0 Disabled Repeat this for slots 1-5 (the last slot is off the bottom of the screen). You should now have: Scanning OPROM on PCI-E Slot0 Disabled Scanning OPROM on PCI-E Slot1 Disabled Scanning OPROM on PCI-E Slot2 Disabled Scanning OPROM on PCI-E Slot3 Disabled Scanning OPROM on PCI-E Slot4 Disabled Scanning OPROM on PCI-E Slot5 Disabled The next actions depend on the type external storage and the HBAs. The SAS2 HBA slot restrictions require a different combination of Enabled/Disabled. Take care to use the appropriate settings. For SAS1 configurations (Connected with J4400 JBODs) Just below these OPROM settings (they are actually off the bottom of the screen and you will need to scroll down) are a group of settings which allow IO allocation to be disabled per-slot. Disable PCI-E slots 0-3, checking that slots 4 and 5 are Enabled. It should look like: IO Allocation on PCI-E Slot0 Disabled IO Allocation on PCI-E Slot1 Disabled IO Allocation on PCI-E Slot2 Disabled IO Allocation on PCI-E Slot3 Disabled IO Allocation on PCI-E Slot4 Enabled IO Allocation on PCI-E Slot5 Enabled For SAS2 configurations (Connected with J4410 JBODs) Just below these OPROM settings (they are actually off the bottom of the screen and you will need to scroll down) are a group of settings which allow IO allocation to be disabled per slot. Disable PCI-E slots 1-4, checking that slots 0 and 5 are Enabled. It should look like: IO Allocation on PCI-E Slot0 Enabled IO Allocation on PCI-E Slot1 Disabled IO Allocation on PCI-E Slot2 Disabled IO Allocation on PCI-E Slot3 Disabled IO Allocation on PCI-E Slot4 Disabled IO Allocation on PCI-E Slot5 Enabled ---------------------------------- END OF 'IO Allocations' STEP ------------------------------------ Next, arrow over to the Boot menu. Select the last item: "Hard Disk Drives" and press return. The list should include only 2 drives (the 2 internal SATA drives) with labels like: SATA:11M- SATA:12M- If this list includes anything else (such as readzilla cache devices with a 'STEC MACH8' string, or JBOD attached drives) you'll need to remove them from the list by selecting the boot position and setting it to 'Disabled' for each of non-boot drives. If the list is full (with 16 drives) you will not be able to edit the list. However, the change to the OPROM settings above will cause the JBOD drives to disappear from the list on the next boot. You will need to exit and save changes and immediately re-enter the BIOS menu on the next boot (CTRL-E). Exiting BIOS Setup Once you've removed any readzilla cache or JBOD drive entries from the "Hard Disk Drives" list, press ESC to exit the "Hard Disk Drives" menu, then arrow right to the "Exit" menu. Press for the default "Save Changes and Exit", and return again to confirm the action with the pop up. On boot, you will see the following warning message from the BIOS: Warning: IO resource not allocated This is an expected message and does not indicate a failure.
Finally, resync the SP password to match the root password of the NAS head. Have the customer complete this final step. Remember: you exit back to the SP using ESC-(
-> cd /SP/users/root
/SP/users/root -> set password Enter new password: ********* Enter new password again: *********
With the SP password matching the root password, the motherboard replacement is complete. NOTE: The SP has a minimum password length of 8 which is not enforced by the appliance system software. ie, if the customer has the hopelessly simple password "abc", then this will be rejected by the SP. To resolve this, the customer will need set a new password from the appliance, which in turn will update the SP password directly. Attachments This solution has no attachment |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||