bdachecknet to Verify the BDA Network Configuration Fails with Reverse Lookup Errors

Asset ID:	1-72-1923938.1
Update Date:	2017-11-02
Keywords:

Solution Type Problem Resolution Sure

Solution 1923938.1 : bdachecknet to Verify the BDA Network Configuration Fails with Reverse Lookup Errors

Applies to:

Big Data Appliance X3-2 Hardware - Version All Versions and later
Big Data Appliance X4-2 Hardware - Version All Versions and later
Linux x86-64

Symptoms

Running bdachecknet to verify the network configuration fails with:

# bdachecknet

bdachecknet: analyse /opt/oracle/bda/BdaDeploy.json
bdachecknet: passed
bdachecknet: checking for BdaExpansion.json
bdachecknet: ping test private infiniband ips (bondib0 40gbs)
bdachecknet: passed
bdachecknet: ping test admin ips (eth0 1gbs)
bdachecknet: passed
bdachecknet: test admin network resolve and reverse resolve
bdachecknet: passed
bdachecknet: test admin name array matches ip array
bdachecknet: passed
bdachecknet: test client network (eoib) resolve and reverse resolve
bdachecknet: ip to name to ip resolve failed for ip address: 10.*.*.22 (!= )
bdachecknet: ip to name to ip resolve failed for ip address: 10.*.*.23 (!= )
bdachecknet: ip to name to ip resolve failed for ip address: 10.*.*.24 (!= )
bdachecknet: ip to name to ip resolve failed for ip address: 10.*.*.25 (!= )
bdachecknet: ip to name to ip resolve failed for ip address: 10.*.*.26 (!= )

1. A reverse lookup on one of the failing IPs fails with an error as below:

$ nslookup 10.*.*.22

Got SERVFAIL reply from <nameserver ip>, trying next server
...

** server can't find 22.*.*.10.in-addr-arpa.: NXDOMAIN

Note: It is possible that nslookup on an IP may succeed because it is using the DNS servers specified in /etc/resolv.conf.

2. Running "host <IP>" to manually perform a reverse lookup on the IP also fails with a SERVFAIL error.

$ host 10.*.*.22

Host 22.*.*.10.in-addr.arpa not found: 2(SERVFAIL)

The same failure occurs if running "host <IP> <nameserver IP>". For example:

$ host 10.*.*.22 <nameserver IP>

Using domain server:
Name: <nameserver>
Address: ...
Aliases:

Host 22.*.*.10.in-addr.arpa not found: 2(SERVFAIL)

3. However, each Client IP can pinged. In other words, the following succeeds for each Client IP:

$ ping -c 2 <each Client IP>

Cause

bdachecknet attempts a reverse resolve on Client IP addresses. The expectation is that a reverse lookup on a Client IP will be resolved.

The bdachecknet error: "ip to name to ip resolve failed for ip address: *.*.*.* (!= ) indicates that Client IP addresses are not resolving i.e. the reverse DNS lookup on the Client IP is failing. There is a problem with reverse lookup at the site.

Solution

Correct the problem with reverse lookup on the site. For example the Client IP may not be in the DNS server.

Once the underlying issue preventing DNS reverse lookups on Client IPs is resolved bdachecknet will successful pass this check.

If the problems with reverse lookups at the customer site can not be resolved, they can be ignored if no other errors. A reverse lookup failure will not prevent
bdacheckcluster from succeeding and will not cause an upgrade failure.

References

<NOTE:1923949.1> - Zookeeper Will Not Start on One BDA Node Due to Error Reading Snaphot Files

Attachments

This solution has no attachment