Exachk Fails to Verify the Password for Host or Switch Even if the Password is Correct

Asset ID:	1-72-1515124.1
Update Date:	2016-09-05
Keywords:

Solution Type Problem Resolution Sure

Solution 1515124.1 : Exachk Fails to Verify the Password for Host or Switch Even if the Password is Correct

Applies to:

Exadata Database Machine X2-2 Full Rack - Version All Versions to All Versions [Release All Releases]
Oracle Exadata Hardware - Version 11.2.0.1 and later
Oracle Solaris Express
Linux x86-64

Symptoms

You are running Exachk to verify the configuration of the Rack, but it fails on the password for either a server or Infiniband switch.

You verify the following:

1. You are able to login to the IB switch (or server) from remote shell:

# ssh a4xbsw-ib1
root@a4xbsw-ib1's password:
Last login: Fri Dec 14 05:21:50 2012 from 10.223.196.24
#

2. The exachk script is failing with same password. The password is confirmed to be alphanumeric:

9 of the included audit checks require root privileged data collection on INFINIBAND SWITCH .

1. Enter 1 if you will enter root password for each INFINIBAND SWITCH when prompted

2. Enter 2 to exit and to arrange for root access and run the exachk later.

3. Enter 3 to skip checking best practices on INFINIBAND SWITCH

Please indicate your selection from one of the above options[1-3][1]:-

Is root password same on all INFINIBAND SWITCH ?[y/n][y]

Enter root password for INFINIBAND SWITCH :-

root password for a4xbsw-ib1 was incorrect. 2 retries remaining.

Enter root password for a4xbsw-ib1 :-

root password for a4xbsw-ib1 was incorrect. 1 retries remaining.

Enter root password for a4xbsw-ib1 :-

root password for a4xbsw-ib1 was incorrect. root privileged checks will not be executed on a4xbsw-ib1

root passwords for following nodes are incorrect.

3. Execute exachk in debug mode as follows:

# script /tmp/exachk.out
# sh -x exachk
<follow the steps, reproduce your problem>
# <press CTRL-D>

From the debug output, you can see the password was correctly saved, but "expect" command returns "2":

+ passwordToCheck=<password input> <<<<<<<<<<<<<
+ /usr/bin/expect -f -
++ echo 2 <<<<<<<<<<<<<<<<<<
+ passwordCheckStatus=2

4. Enable expect debug by setting "RAT_EXPECT_DEBUG", and execute "exachk" again.

# export RAT_EXPECT_DEBUG="-d"
# exachk

"expect:timed out" is explicitly reported during password verification.

parent: waiting for sync byte
parent: telling child to go ahead
parent: now unsynchronized from child
spawn: returns {58168}

expect: does "" (spawn_id exp6) match glob pattern "no)?"? no
"*?assword:*"? no
expect: timed out

expect: does "" (spawn_id exp6) match glob pattern "*?assword:*"? no
expect: timed out

Cause

It is taking too long to get the password prompt from this particular server or IB switch. The default timeout is 1 second.

This problem is confirmed in version 2.2.0 or older.

Solution

Try to get the approximate time that password prompt appears in screen using the following test:

# ssh -q root@<hostname> "echo \"LoginSuccessfull\""

Now before executing exachk, set RAT_PASSWORDCHECK_TIMEOUT to a higher value than what you observed in last step, for example, 40:

# export RAT_PASSWORDCHECK_TIMEOUT=40
# exachk

This should allow enough time to finish the Exachk collection.

The time delay, however, should be investigated to determine if excessive.

References

<BUG:16032818> - UNABLE TO VERIFY ROOT PASSWORD FOR IB SWITCH
<NOTE:1070954.1> - Oracle Exadata Database Machine exachk or HealthCheck

Attachments

This solution has no attachment