Sun SPARC Enterprise M3000/M4000/M5000/M8000/M9000 Servers: some insights for checking domain's console connection

Asset ID:	1-79-1906225.1
Update Date:	2018-05-09
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 1906225.1 : Sun SPARC Enterprise M3000/M4000/M5000/M8000/M9000 Servers: some insights for checking domain's console connection

Applies to:

Sun SPARC Enterprise M3000 Server
Sun SPARC Enterprise M4000 Server
Sun SPARC Enterprise M5000 Server
Sun SPARC Enterprise M8000 Server
Sun SPARC Enterprise M9000-32 Server
Information in this document applies to any platform.

Purpose

The document gives some insights about some commands and settings that are relevant for a successful domain console
connection. In practice there might be a problematical console as such that the console connection for a domain is not
possible anymore, or the console connection cannot be established reliably and is not readable. In general the reason
for this can be caused by a wrong configuration on domain's Solaris OS or a faulty hardware component. To connect to
a questionable domain for troubleshooting purposes a connection over network can be used.

Scope

Details

1) An "OPL serial mux driver" is used for the console on domain's Solaris OS. In case this driver is not present it can
happen that the console does not respond anymore after entering the password and pressing return though the domain is
in fact not hanging but its console not responding. The following outlines what should exist analogue on each domain.

   ha-m3000a-n1:(/)# grep iu.ap /etc/inittab
   ap::sysinit:/usr/sbin/autopush -f /etc/iu.ap

   ha-m3000a-n1:(/)# cat /etc/iu.ap
   #
   # sparc /dev/console and /dev/contty autopush setup
   #
   #       major minor   lastminor       modules
           [...]
           oplmsu 0       0       ldterm ttcompat

   ha-m3000a-n1:(/)# ls -al /usr/sbin/autopush
   -r-xr-xr-x   1 root     bin        21092 Jun 12 17:10 /usr/sbin/autopush

   ha-m3000a-n1:(/)# modinfo |egrep "addr|oplmsu"
    Id Loadaddr   Size Info Rev Module Name
    93 7b7a6000   8bc0 154   1 oplmsu (OPL serial mux driver)

The three files mentioned above are also mentioned in Document# 1010145.1 (How to troubleshoot staircase text at bootup
(and problem logging into single user)).

Btw: As per ttymon(1M) (port monitor for terminal ports) the invocation of ttymon on the system console is managed
under smf(5) by the service svc:/system/console-login. You can check the service console-login:default as outlined
below to see that the service is online and to see its properties. The example below outlines the property group ttymon.

   ha-m3000a-n1:(/)# svcs console-login:default
   STATE          STIME    FMRI
   online         Jul_07   svc:/system/console-login:default

   ha-m3000a-n1:(/)# svccfg -s svc:/system/console-login:default listprop ttymon
   ttymon                application
   ttymon/device        astring     /dev/console
   ttymon/terminal_type astring     xterms

2) Sometimes it is the TTY setting that causes a non readable console. You can use sttydefs(1M) to inquire and set the
right settings for the ttylabel "console". If required you can remove (-r) and add (-a) the ttylabel again with the
right settings which will be reflected in the file /etc/ttydefs as well.

   ha-m3000a-n1:(/)# sttydefs -r console
   ha-m3000a-n1:(/)# sttydefs -a console -i "9600 hupcl opost onlcr crtscts" -f "9600"

   ha-m3000a-n1:(/)# sttydefs -l console |grep .
   ----------------------------------------------------
   console:9600 hupcl opost onlcr crtscts:9600::console
   ----------------------------------------------------
   ttylabel:       console
   initial flags: 9600 hupcl opost onlcr crtscts
   final flags:    9600
   autobaud:       no
   nextlabel:      console

   ha-m3000a-n1:(/)# grep console: /etc/ttydefs
   console:9600 hupcl opost onlcr crtscts:9600::console

3) A possible cause for a corrupt terminal output on the console might be found and fixed by stty(1) and by ensuring
certain options (control modes) for the terminal. Please note that some settings on a given domain might differ from
the example below while the console is working fine. The most relevant options are outlined with the first stty command
below. The second stty command is an example that shows an inquiry of all of the option settings for the terminal.
The options are described in the man page stty(1).

   ha-m3000a-n1:(/)# stty -parenb cs8 -ignpar -istrip tab3
   ha-m3000a-n1:(/)# stty -a < /dev/console
   speed 9600 baud;
   rows = 0; columns = 0; ypixels = 0; xpixels = 0;
   csdata ?
   eucw 1:0:0:0, scrw 1:0:0:0
   intr = ^c; quit = ^\; erase = ^?; kill = ^u;
   eof = ^d; eol = <undef>; eol2 = <undef>; swtch = <undef>;
   start = ^q; stop = ^s; susp = ^z; dsusp = ^y;
   rprnt = ^r; flush = ^o; werase = ^w; lnext = ^v;
   -parenb -parodd cs8 -cstopb hupcl cread -clocal -loblk crtscts -crtsxoff -parext
   -ignbrk brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl -iuclc
   ixon -ixany -ixoff imaxbel
   isig icanon -xcase echo echoe echok -echonl -noflsh
   -tostop echoctl -echoprt echoke -defecho -flusho -pendin iexten
   opost -olcuc onlcr -ocrnl -onocr -onlret -ofill -ofdel tab3

4) Finally there is chance that a botched domain console is caused by a faulty hardware component. A faulty hardware
component could be the reason even if there are no indications as per XCSF's 'showlogs' so far. To be safe a reboot/POST
of the relevant domain is required which might reveal the cause. See the following example of an M4000 System (Firmware
XCP1093) where the problematical hardware component MBU_A (Systemboard) is found only with an initiated POST some days
after encountering the trouble. The MBU_A needs to be replaced.

   XSCF> showlogs console -d 0
   DomainID: 00
   May 24 14:44:03 UTC 2014      ~Xà
   [...]
   May 26 11:10:36 UTC 2014      f~Føf
   May 26 20:43:02 UTC 2014      ~^~^~F~^f^Ff^^
   XSCF>

   XSCF> showlogs monitor
   [...]
   Jan 25 10:23:49 <hostname> monitor_msg: SCF:DomainID 0 state change (system running, detail#10)
   Jun 28 08:07:48 <hostname> monitor_msg: SCF:DomainID 0 state change (shutdown started, detail#2)
   Jun 28 08:07:50 <hostname> monitor_msg: SCF:DomainID 0 state change (boot process started, detail#9)
   Jun 28 08:08:30 <hostname> monitor_msg: SCF:DomainID 0 state change (initialize phase started, detail#9)
   Jun 28 08:08:39 <hostname> monitor_msg: SCF:DomainID 0: Reset released
   Jun 28 08:08:52 <hostname> Warning: /MBU_A#0:POST:Tty data compare error
   Jun 28 08:08:52 <hostname> monitor_msg: SCF:Current domains' phase (DomainID 0 domain phase: CPU Check)
   [...]
   Jun 28 08:14:23 <hostname> monitor_msg: SCF:Current domains' phase (DomainID 0 domain phase: OpenBoot Start...)
   Jun 28 08:14:25 <hostname> monitor_msg: SCF:Domain issued power-off request to RCI target (DomainID 0)
   Jun 28 08:14:27 <hostname> Warning: /UNSPECIFIED:RESET:All console paths in domain failed
   Jun 28 08:14:29 <hostname> monitor_msg: SCF:All domains shutdown started
   Jun 28 08:14:39 <hostname> monitor_msg: SCF:DomainID 0 state change (Powered off, detail#8)
   Jun 28 08:15:34 <hostname> monitor_msg: SCF:System powered off
   Jun 28 08:17:37 <hostname> monitor_msg: SCF:System powered on
   [...]

   XSCF> showlogs -v error
   Date: Jun 28 08:08:49 UTC 2014     Code: 6000a000-d5010000-0200250200000000
       Status: Warning                Occurred: Jun 28 08:08:47.764 UTC 2014
       FRU: /MBU_A#0
       Msg: Tty data compare error
       Diagnostic Code:
           00000000 00000000 00000000
           00000000 00000000 00000000 00000000
           00000000 00000000 00000000 00000000
       UUID: 57f3fb16-2337-48a5-bc4e-a74330553178 MSG-ID: SCF-8003-LS
   Date: Jun 28 08:14:26 UTC 2014     Code: 6000c000-ffff0000-0400003800000000
       Status: Warning                Occurred: Jun 28 08:14:25.566 UTC 2014
       FRU: /UNSPECIFIED
       Msg: All console paths in domain failed
       Diagnostic Code:
           ffffffff ffff0000 00000000
           00000000 00000000 00000000 00000000
           00000000 00000000 00000000 00000000
       UUID: 5a95a814-06de-4040-93cb-b3d276079c35 MSG-ID: SCF-8008-3U

   XSCF> showstatus
   *   MBU_A Status:Degraded;

See the following bug as an example for a corrupted terminal output caused by Software:
BUG# 15699614 / SUNBT7023040 (28-Feb-2011) stty sane corrupts the terminal output

References

<NOTE:1010145.1> - How to troubleshoot staircase text at bootup (and problem logging into single user).

Attachments

This solution has no attachment