FS System - Serial Connection Doesn't Work After Pilot MB Replacement

Asset ID:	1-72-2232655.1
Update Date:	2017-02-18
Keywords:

Solution Type Problem Resolution Sure

Solution 2232655.1 : FS System - Serial Connection Doesn't Work After Pilot MB Replacement

Applies to:

Oracle FS1-2 Flash Storage System - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

The FS1 GUI overall system state will be WARNING and both Pilots will report the Serial Connection as OFFLINE/MISSING.

Changes

This has been observed to occur after a Pilot motherboard replacement.

Cause

The ILOM configuration, specifically the /SP/serial/portsharing setting, for the Pilot motherboard has not been properly configured.

Solution

Confirm the ILOM portsharing Setting

Enable ssh to the Pilot, reference KM doc 2029847.1 FS System: How to Enable SSH to the Pilot and ssh to the active Pilot.
ssh to the ILOM of the Pilot who's motherboard was replaced (169.254.2.5 if starting from that Pilot, 169.254.2.9 if not) password = changeme.
Display the portsharing owner:

-> show /SP/serial/portsharing

/SP/serial/portsharing
    Targets:

    Properties:
        owner = sp

    Commands:
        cd
        set
        show

->
If the owner value is sp, proceed to the next section. If it is host, the problem is elsewhere.

Correct the ILOM portsharing Setting

An FE will be needed to assist as a direct connect to the SER_MGT port of the Pilot with the problem is required to make the change(s).

Order a spare Pilot Motherboard just in case.

Make sure that the active Pilot is not the Pilot with the replacement motherboard. If it is, force a failover using fscli:

# fscli pilot -forceFailover
Disconnect the serial cable from the SER_MGT port of the problem Pilot.
Connect a NULL Modem cable between the RJ45 SER MGMT port of that Pilot and a laptop.
Establish connection using the following settings in PuTTY (or similar application):
1. 9600 baud
2. 8+1+N, 8 data bits, 1 stop bit, no parity
3. No flow control
4. Appropriate COMM port on the laptop
Hit return to get a login prompt and login with root/changeme as user/password
Change the portsharing value from sp to host:

-> show /SP/serial/portsharing

/SP/serial/portsharing
    Targets:

    Properties:
        owner = sp

    Commands:
        cd
        set
        show

-> set /SP/serial/portsharing/owner=host

-> show /SP/serial/portsharing

/SP/serial/portsharing
    Targets:

    Properties:
        owner = host

    Commands:
        cd
        set
        show

->
Reset the ILOM and have the FE reconnect the SERIAL connections between the 2 Pilots.

ILOM> reset /SP
After the green SP LED in front of the Pilot goes solid, run the following three commands in succession to ensure the other ILOM settings are correct (in this example Pilot 2 had its motherboard replaced so the commands are run from Pilot 1):
[root@pilot1 ~]# ubiosconfig import all -H 169.254.2.9 -U root -yx /rpms/firmware/<Pilot-model-specific-bios-defaults.xml>
[root@pilot1 ~]# ilomconfig import config -H 169.254.2.9 -U root -y --xmlfile=/rpms/firmware/<Pilot-model-specific-ilom-defaults.xml>
[root@pilot1 ~]# fwupdate update sp-bios-firmware -n sp_bios -f /rpms/firmware/<Pilot-model-specific-ILOM.pkg> -o=/var/log/messages -q -H=169.254.2.9 -U=root

If any of the above commands errors out, you can find the error in the ILOM by running "show /SP/logs/event/list", and checking the top most lines for the most recent events. If there are errors, repeat steps 7 and 8.
Once the above commands are completed and there are no errors, check to make sure that the BIOS version do still match
[root@pilot1 ~]# ipmitool -H 169.254.2.5 -U root -P changeme sunoem cli "version" -I lanplus
Connected. Use ^D to exit.
-> version
SP firmware 3.1.2.30.b
SP firmware build number: 86439
SP firmware date: Mon Jan 20 17:03:25 CST 2014
SP filesystem version: 0.1.23'

[root@pilot1 ~]# ipmitool -H 169.254.2.9 -U root -P changeme sunoem cli "version" -I lanplus

Connected. Use ^D to exit.
-> version
SP firmware 3.1.2.30.b
SP firmware build number: 86439
SP firmware date: Mon Jan 20 17:03:25 CST 2014
SP filesystem version: 0.1.23'
Reboot the SP once more to ensure all changes are permanent
ILOM> reset /SP
Once the ILOM is back, confirm that the FS1 GUI is now reporting all as normal.

If there are problems with any portion of the procedure, and resetting the /SP does not resolve the problem, it is recommended to replace the motherboard again.

Attachments

This solution has no attachment