bdacheckcluster Raises: "ERROR: 2 virtual NICs not available on host bdanodex" when VNIC Created on IB2 is in WAIT-IOA State without Node Information

Asset ID:	1-72-2112934.1
Update Date:	2018-05-28
Keywords:

Solution Type Problem Resolution Sure

Solution 2112934.1 : bdacheckcluster Raises: "ERROR: 2 virtual NICs not available on host bdanodex" when VNIC Created on IB2 is in WAIT-IOA State without Node Information

Applies to:

Big Data Appliance Hardware - Version All Versions and later
Big Data Appliance Integrated Software - Version 4.0 and later
Linux x86-64

Symptoms

The symptoms are that after the cable between the switch bda1sw-ib2 and Port 2 of the BDA server is replaced due to Port 2 being down as indicated by ibstat the following are observed:

1.bdacheckcluster fails with: ERROR: 2 virtual NICs not available on host bdanodex.

# bdacheckcluster reports:

ERROR: 2 virtual NICs not available on host bdanodex

2. ifup eth9 raises: Device eth9 does not seem to be present, delaying initialization.

# ifup eth9

Device eth9 does not seem to be present, delaying initialization.

3. Running /opt/oracle/bda/network/BdaUserConfigEoib fails to resolve the issue.

4. However "showvnics" shows that the node is not using both ports even though both ports are now available.

#showvnics

51 WAIT-IOA N <GUID> 0000 <MAC&> NO ffff 0A-ETH-1

The VNIC created in IB3 is up and running, but the VNIC created on IB2 is in WAIT-IOA state and there is no node information:

ID STATE FLG IOA_GUID NODE IID MAC VLN PKEY GW
--- -------- --- ----------------------- -------------------------------- ---- ----------------- --- ---- --------

51 WAIT-IOA N <GUID> 0000 <MAC&> NO ffff 0A-ETH-1

Changes

This happens after replacing the cable between the switch bda1sw-ib2 and Port 2 of the BDA server because ibstat showed that Port 2 was down.

Cause

The WAIT-IOA state indicates the node is not responding. VNICs staying in WAIT-IOA indicates that the defined VNIC on the Gateway is unable to communicate with the IB Port.

Some reasons for this are:

1. The node not reachable. (This is not the case here, the node is reachable.)
2. The node does not support EoIB. (This is not the case here.)
3. The node does not have the driver loaded. (This is not likely.)
4. The node is not reachable via the guid configured. It appears node is not responding to traffic received on this guid. (This does look to be the case here.)

Hence it looks like Linux is refusing to respond on port 2 even though it is now up and running.

Solution

One option is to reboot of the node with where bdacheckcluster reports the error.

Then from switch bdasw-ib2 run:

# showvnics

This should now confirm all states are UP. bdacheckcluster should no longer report a VNIC not available error.

Attachments

This solution has no attachment