How to force a crash dump on SPARC T5xx0/T3-x/T4-x/T5-x/T7-x/T8-x/Sun Blade T6320/T6340 (with iLOM shell)

Asset ID:	1-71-1392395.1
Update Date:	2018-01-26
Keywords:

Solution Type Technical Instruction Sure

Solution 1392395.1 : How to force a crash dump on SPARC T5xx0/T3-x/T4-x/T5-x/T7-x/T8-x/Sun Blade T6320/T6340 (with iLOM shell)

Applies to:

SPARC T3-4 - Version Not Applicable and later
SPARC T3-1 - Version Not Applicable and later
Sun SPARC Enterprise T5140 Server - Version Not Applicable and later
Sun SPARC Enterprise T5240 Server - Version Not Applicable and later
Sun SPARC Enterprise T5440 Server - Version Not Applicable and later
Oracle Solaris on SPARC (64-bit)
ILOM, SPARC T5xx0, SPARC T3-x, SPARC T4-x, SPARC T5-x, SPARC T7-x, panic, crash dump, hang

Goal

This document provides a description/usage of how to force a kernel crash dump through the ILOM shell on SPARC T5xx0/T3-x/T4-x/T5-x/T7-x/T8-x/Sun Blade T6320/Sun Blade T6340 series of machines (including their Netra variants).

For SPARC T5xx0 that is using ALOM shell, please refer to Doc Id 1004092.1 "How to force a crash dump on SunFire[TM] T1000/T2000"

For general guidelines on "How to force a crash dump when the OS is hung", please refer to doc 1004506.1.

Solution

Steps to Follow

In order to force a crash dump on SPARC T5xx0/T3-x/T4-x/T5-x/T7-x/T8-x/Sun Blade T6320/T6340 under ILOM, please follow the following steps:

When the system is hung, a kernel crash dump is needed to identify the cause of the hang.

1. The keyswitch state needs to be out of "diag" or "lock" position prior to a force reset. In order to set the keyswitch to "normal", login to the SP. At the ILOM's "->" prompt, issue the following command:

-> set /SYS keyswitch_state=normal
Set ’keyswitch_state’ to ’normal’

2. Once the keyswitch_state is in normal mode, issue the following commands to force a dump and get back to the console to monitor the coredumping process.

-> set /HOST send_break_action=dumpcore
-> start /SP/console

(On SPARC T5xx0, once login to the SP, if the prompt is shown as "sc> ", the login is using ALOM interface. In this case, please refer to Doc Id 1004092.1)

3. Once the system is booted up, the collected core dump would be in /var/crash/<hostname>/vmdump.<sequence id>.

Example:

	-> set /SYS keyswitch_state=normal
	Set ’keyswitch_state’ to ’normal’ 
	-> set /HOST send_break_action=dumpcore
	Set 'send_break_action' to 'dumpcore'
	-> start /SP/console
	Are you sure you want to start /SP/console (y/n)? y

	Serial console started.  To stop, type #.
	 done
	dumping to /dev/dsk/c0t5000CCA00AB4C674d0s1, offset 107544576, content: kernel
	 0:50  100% done
	 ...

	rebooting...

NOTE: To return from the system console, back to the SP -> prompt, please type #. (i.e. a # following by dot).

Alternatively, one can also send break via the following command and getting back to the console to "sync" to force a dump:

-> set /HOST send_break_action=break 
-> start /SP/console
Serial console started.  To stop, type #.

c)ontinue, s)ync, r)eset? s

A NOTE ON ALTERNATE BREAK SEQUENCE

If the system has been setup to use Alternate Break sequence, i.e. having the following entry effective in /etc/default/kbd:

    # Uncomment the following line to enable a non-BREAK alternate
    # serial input device abort sequence:
       KEYBOARD_ABORT=alternate

the system will not respond to the send break sequence from the iLOM (or ALOM). i.e. "set /HOST send_break_action=break" will not work. If this is the setup, please use the Alternate Break sequence, by getting onto the system console and type "~" follow with a CTRL-B:

    -> start /SP/console
    Are you sure you want to start /SP/console (y/n)? y

    Serial console started. To stop, type #.
    <"~" & "CTRL-B">               <---- Key sequence from Keyboard. Will not be visible on screen.
    Debugging requested; hardware watchdog suspended.
    c)ontinue, s)ync, r)eset?

Once at the "c)ontinue, s)ync,r)eset?" prompt, type "s" to force a dump.

For more information regarding break sequence, please refer to kdb(1) man page.

Alternate Break Sequence *DOES NOT* affect "set /HOST send_break_action=dumpcore".

KEYBOARD_ABORT variable in /etc/default/kbd can have values of enable (default), disable or alternate.

- if KEYBOARD_ABORT=enable, then '-> set /HOST send_break_action=break' from ILOM CLI, would work
- if KEYBOARD_ABORT=disable, then '-> set /HOST send_break_action=break' from ILOM, would not work.
- if KEYBOARD_ABORT=alternate, then '-> set /HOST send_break_action=break' from ILOM, would not work, however the alternate break sequence will work, as described below.
- Note: set /HOST send_break_action=dumpcore in ILOM always work, irrespective of value of KEYBOARD_ABORT

Example:

	-> set /SYS keyswitch_state=normal
	Set ’keyswitch_state’ to ’normal’ 	
	-> set /HOST send_break_action=break 
	Set 'send_break_action' to 'break'
	-> start /SP/console
	 Are you sure you want to start /SP/console (y/n)? y 

	Serial console started.  To stop, type #.

	c)ontinue, s)ync, r)eset? s
	panic[cpu53]/thread=2a1014b1ca0: sync initiated
	sched: trap type = 0x0
	pid=0, pc=0x0, sp=0x0, tstate=0x0, context=0x0
	o0-o7: 0, 0, 0, 0, 0, 0, 0, 0
	g1-g7: 0, 0, 0, 0, 0, 0, 0
	000002a1014b1390 unix:sync_handler+150 (1d, 10b9800, 35, 3001507a000, 1, 183f800)
	  %l0-3: 0000000000000000 0000000001913800 0000000000000000 000000000190d000
	  %l4-7: 0000000000000037 000000000183a400 0000000000000000 000003001507e000
	000002a1014b1460 unix:vx_handler+80 (2a1014b15c0, 184ac98, 19daef8, 1, 184ada0, 1847ba8)
	  %l0-3: 000000000184ada0 0000000000000000 0000000000000001 0000000000000001	
	  %l4-7: 000000000183ac00 0000000001000000 0000000001000000 000000000101cf9c
	000002a1014b1510 unix:promif_enter_mon+14c (1847800, 10bdc00, 10bdc00, 73, 10bdc90, 2a1014b15c0)
	  %l0-3: 000000000183d1a8 0000000001847ba8 00000000010bdc80 00000000010bdc78
	  %l4-7: 00000000010bdc70 000000000000000c 0000000001000000 0d00000000000000
	000002a1014b15d0 unix:kern_cif_handler+28 (2a1014b17e8, 10bd800, 18480c0, 0, 1, 104e53c)
	  %l0-3: 000002a1014b1634 0000000000000bfd 00000000000003ff 0000000000000ffc
	  %l4-7: 00000000000003fe 8000000000000000 0000000080000000 8000000000000000
	000002a1014b1680 unix:client_handler+2c (104db9c, 2a1014b17e8, 0, 184ac90, 1918000, 104db9c)
	  %l0-3: 000000000190d2f0 0000000000000001 000000000183ac00 0000000000000001
	  %l4-7: 0000000000000016 000000000000000e 0000000000000016 000003001507a000
	000002a1014b1730 unix:prom_enter_mon+24 (0, 0, 18ef000, 1, 183d390, 18480c0)
	  %l0-3: 0000000001938510 000006009541cfc2 0000000000000000 000000000193e000
	  %l4-7: 0000000000000001 000000000193e000 0000000000000000 0000000000000070
	000002a1014b1800 unix:debug_enter+118 (0, a, a, 183a400, 0, 190d000)
	  %l0-3: 0000000000000001 00000600a1b72ca0 0000000000000000 0000000000000007
	  %l4-7: 00000600a94cf6e0 00000000010e4864 000006009c35b440 00000600a1b72cf0
	000002a1014b18d0 unix:abort_seq_softintr+94 (183a400, 198f400, 3001507a000, 2a1014b1d78, 1, 193d000)
	  %l0-3: 0000000001849e20 00157dff507475b1 0000000000000000 0000000000000000
	  %l4-7: 0000000000000000 000000000198f400 0000000000000000 0000000001010f3c

	syncing file systems...

1392395.1

References

<NOTE:1004092.1> - How to force a crash dump on SunFire[TM] T1000/T2000
<NOTE:1004506.1> - How to Force a Crash Dump When the Solaris Operating System is Hung

Attachments

This solution has no attachment