Running networksetup-one Fails on One Node in Cluster while Installing X4-2 Oracle Big Data Appliance

Asset ID:	1-72-2050261.1
Update Date:	2015-09-01
Keywords:

Solution Type Problem Resolution Sure

Solution 2050261.1 : Running networksetup-one Fails on One Node in Cluster while Installing X4-2 Oracle Big Data Appliance

Applies to:

Big Data Appliance X4-2 Hardware - Version All Versions and later
Linux x86-64

Symptoms

When installing an X4-2 Oracle Big Data Appliance the networksetup-one script fails on one node of the cluster, like node 5, with:

[root@bda01 network]# ./networksetup-one
networksetup-one: analyse /opt/oracle/bda/BdaDeploy.json
networksetup-one: passed
networksetup-one: checking for BdaExpansion.json
networksetup-one: ping servers on ship admin network by ip
networksetup-one: passed
networksetup-one: test ssh to servers on ship admin network by ip
networksetup-one: passed
networksetup-one: copy /opt/oracle/bda/BdaDeploy.json to servers by ip
networksetup-one: verify transfer of /opt/oracle/bda/BdaDeploy.json
networksetup-one: passed
networksetup-one: executing network settings on all servers
networksetup-one: wait a few seconds for the network to restart on <factory IP>
networksetup-one: wait a few seconds for the network to restart on <factory IP>
networksetup-one: wait a few seconds for the network to restart on <factory IP>
networksetup-one: wait a few seconds for the network to restart on <factory IP>
networksetup-one: wait a few seconds for the network to restart on <factory IP>
networksetup-one: bdanode02.example.com BdaUserConfigNetwork: reset network
networksetup-one: bdanode03.example.com BdaUserConfigNetwork: reset network
networksetup-one: bdanode06.example.com BdaUserConfigNetwork: reset network
networksetup-one: bdanode04.example.com BdaUserConfigNetwork: reset network
networksetup-one: Communication error
networksetup-one: deploying this server
networksetup-one: network will restart momentarily, pardon our dust

networksetup-one: bdanode01.example.com BdaUserConfigNetwork: reset network
bonding: bondib0: Removing slave ib0.
bonding: bondib0: releasing active interface ib0
bonding: bondib0: making interface ib1 the new active one.
bonding: bondib0: Removing slave ib1.
bonding: bondib0: releasing active interface ib1
bonding: bondib0: setting mode to active-backup (1).
bonding: bondib0: Setting MII monitoring interval to 100.
bonding: bondib0: Note: Updating updelay (to 5000) since it is a multiple of the miimon value.
bonding: bondib0: Note: Updating downdelay (to 5000) since it is a multiple of the miimon value.
bonding: bondib0: Setting down delay to 5000.
bonding: bondib0: Setting up delay to 5000.
8021q: adding VLAN 0 to HW filter on device bondib0
ib0: enabling connected mode will cause multicast packet drops
ib0: mtu > 2044 will cause multicast packet drops.
bonding: bondib0: Adding slave ib0.
bonding: bondib0: Warning: enslaved VLAN challenged slave ib0. Adding VLANs will be blocked as long as ib0 is part of bond bondib0
bonding: bondib0: Warning: The first slave device specified does not support setting the MAC address. Setting fail_over_mac to active.
bonding: bondib0: enslaving ib0 as a backup interface with a down link.
ib1: enabling connected mode will cause multicast packet drops
ib1: mtu > 2044 will cause multicast packet drops.
bonding: bondib0: link status up for interface ib0, enabling it in 0 ms.
bonding: bondib0: link status definitely up for interface ib0, 4294967295 Mbps full duplex.
bonding: bondib0: making interface ib0 the new active one.
bonding: bondib0: first active interface up!
bonding: bondib0: Adding slave ib1.
bonding: bondib0: Warning: enslaved VLAN challenged slave ib1. Adding VLANs will be blocked as long as ib1 is part of bond bondib0
bonding: bondib0: enslaving ib1 as a backup interface with a down link.
bonding: bondib0: Setting ib0 as primary slave.
bonding: bondib0: link status up for interface ib1, enabling it in 5000 ms.
8021q: adding VLAN 0 to HW filter on device eth0
bonding: bondib0: link status definitely up for interface ib1, 4294967295 Mbps full duplex.
ixgbe 0000:40:00.0: eth0: NIC Link is Up 1 Gbps, Flow Control: RX/TX
networksetup-one: ping servers on deployed admin network by ip
networksetup-one: ping 10.xxx.xxx.xxx (element# 5) failed

Cause

The cause is that the network on one of the nodes did not configure correctly when running the networksetup-one script.

Solution

For the node with the error do the following:

1. Log in to the node (this may be necessary using a serial console or the factory admin IP).
2. Confirm that /opt/oracle/bda/BdaDeploy.json exists and is valid (if not copy it over). It should be there.
3. Manually rerun the first part of network setup on the node by doing:

/opt/oracle/bda/network/BdaUserConfigNetwork -n <node number>

For example for node 5 you would run the following command:

/opt/oracle/bda/network/BdaUserConfigNetwork -n 5

Please note: The steps here are based on the version of the BDA Base Image that is installed. The X4-2 racks come installed with BDA Base Image v2.6.0 which use BdaDeploy.json. The X5-2 racks come with BDA Base Image v4.1.0 or v4.2.0 which use network.json. Using BdaDeploy.json is correct in this case unless the nodes are reimaged with BDA v4.1.0 in which case the new network.json should be used.

4. If that succeeds, confirm that node 1 can be pinged (admin and private IPs) from the node that had the issue.
5. If that succeeds, return to node 1 and confirm that the node can be pinged (admin and private IPs) from node 1.
6. If all of that succeeds, proceed to networksetup-two.

Attachments

This solution has no attachment