"reset /SYS" Does Not Reboot a BDA Server

Asset ID:	1-72-2254933.1
Update Date:	2017-04-14
Keywords:

Solution Type Problem Resolution Sure

Solution 2254933.1 : "reset /SYS" Does Not Reboot a BDA Server

Applies to:

Big Data Appliance X5-2 Hardware - Version All Versions and later
Linux x86-64

Symptoms

A BDA server is found to be down. Cloudera Manager shows the host to be down and it is reported as "red" in Cloudera Manager > Hosts.

Following, How to Power Cycle an Oracle Big Data Appliance Node Using the ILOM when the Node is Not Reachable Using Ping/SSH (Doc ID 1550440.1), it is not possible to power cycle the BDA node through the ILOM Web interface as there is no ILOM Web interface access on the system.

So the steps in, How to Power Cycle an Oracle Big Data Appliance Node Using the ILOM when the Node is Not Reachable Using Ping/SSH (Doc ID 1550440.1), are followed to power cycle the BDA server via the ILOM CLI.

However it is found that logging into the ILOM and trying to reboot the server with:

-> reset /SYS

Are you sure you want to reset /SYS (y/n)? y
Performing hard reset on /SYS
->

does not result in a server reboot. Instead the ILOM prompt returns.

Cause

The root cause is tracked in an internal bug.

BUG 25189505 - X5-2 stop /SYS Refuses to Stop Instead Initiates System Reset

Solution

As a workaround try stopping and starting /SYS manually. It may be found that the power_state is off and a manual stop and start of /SYS power cycles the server.

To do so:

1. Log into the server's ILOM.

2. Run "show /SYS" to confirm if the power_state is off.

-> show /SYS

Output is like below when the power_state is off:

...
Properties:
type = Host System
ipmi_name = /SYS
...
power_state = Off
...

2. Manually stop /SYS:

-> stop -force /SYS

Output is like:

-> stop -force /SYS

Are you sure you want to immediately stop /SYS (y/n)? y

Broadcast message from root@sbdanodex.domain.com
(unknown) at 12:43 ...

The system is going down for halt NOW!

3. Log back into the ILOM via a new shell and manually start /SYS.

-> start /SYS

Output is like:

-> start /SYS

Are you sure you want to start /SYS (y/n)? y
Starting /SYS

4. Once started check that the power_state is 0n:

-> show /SYS

Output is like:

...
Properties:
type = Host System
ipmi_name = /SYS
...
power_state = On
...

5. Start the console and monitor the start up for any problems.

-> start /SP/console

6. Once the failure is raised at the console prompt, open an SR with Oracle Support.

a) Upload the error message seen in the console to the SR.

b) Collect an ILOM snapshot and upload it to the SR. Follow the steps in SRDC - Oracle Big Data Appliance Hardware Issues (Doc ID 1939356.1) for providing an ILOM snapshot on the server where the alert is triggered, using command line interface.

Attachments

This solution has no attachment