After Firmware Upgrade X6240 Blade Becomes Unbootable

Asset ID:	1-72-1476497.1
Update Date:	2018-03-22
Keywords:

Solution Type Problem Resolution Sure

Solution 1476497.1 : After Firmware Upgrade X6240 Blade Becomes Unbootable

Applies to:

Sun Blade X6240 Server Module - Version Not Applicable to Not Applicable [Release N/A]
Information in this document applies to any platform.

Symptoms

After upgrading FW(firmware), BIOS is not loading and the server becomes unbootable.

On the SP (service processor) internal O/S messages below events are being logged:

Jun 20 12:45:19 localhost do_progparts: running prog_bios /dev/shm/blob_sysbios.515 >& /var/log/blob.sysbios
Jun 20 12:45:27 localhost kernel: Mike driver. Built: Nov 2 2010 at 18:34:24
Jun 20 12:45:27 localhost kernel: hostBIOS: probing 8-bit flash, window=00000000 with JEDEC.
Jun 20 12:45:27 localhost kernel: Found: SST 49LF080A
Jun 20 12:45:27 localhost kernel: hostBIOS: Found 1 x8 devices at 0x0 in 8-bit bank
Jun 20 12:45:27 localhost kernel: number of JEDEC chips: 1
Jun 20 12:45:27 localhost kernel: hostBIOS: found 1048576 bytes
Jun 20 12:45:27 localhost kernel: Mike driver loaded
Jun 20 12:45:43 localhost prog_bios: cmd=[flash_bios -v -p /dev/shm/blob_sysbios.515] failed rc=1 <----------
Jun 20 12:45:44 localhost kernel: lpc_reset(): can't get CPLD out of reset state <---------------------------
Jun 20 12:45:44 localhost kernel: Unloading mike: goodbye, cruel world
Jun 20 12:45:44 localhost do_progparts: do_progparts: prog_bios returned 1
Jun 20 12:45:45 localhost do_progparts: finished
Jun 20 12:45:54 localhost power_event: poweron
Jun 20 12:45:56 localhost do_progparts: begin(poweron)
Jun 20 12:45:56 localhost do_progparts: finished

Changes

Firmware update while powered on.

Cause

To program BIOS, the host must be powered off.

Both host and SP access the same BIOS flash chipset via the LPC bus(Low Pin Count), and the access must be mutually exclusive.
When the host is in powered off state, the MCP55 (Media Communications Processor I/O bridge) is prevented from accessing BIOS LPC by the LPC multiplexer.

When upgrading, SP firmware finishes itself then the SP will automatically reset.
After the SP has reset, ILOM will start to upgrade BIOS if host is in the off status.
If the host is turned on while the SP is programming BIOS, the host MCP55 will start to access BIOS flash at the same time and this will cause the BIOS programming to fail and the BIOS image is corrupted.

We require waiting at least three minutes before powering on the host after the SP is rebooted to allow the SP to finish programming the BIOS.

Please refer to the following Change Request:

<BUG: 15801933> Software/sp X6240 - FW upgrade fails - "cmd=[flash_bios -v -p /dev/shm/blob_sysbios.11564] failed rc=1"

Solution

ACTION PLAN:

- Power off the BLade
- Use ILOM GUI or ILOM CLI to upgrade the blade firmware.
- Once the SP is automatically reset, wait at least 5 minutes before powering on the blade host so that the ILOM has enough time to program the BIOS.
- Then power on the blade.

Attachments

This solution has no attachment