SDM 7.5 Cluster Configuration : Adding a New Cluster gives Error "No route to Host"

Asset ID:	1-72-2131470.1
Update Date:	2017-06-06
Keywords:

Solution Type Problem Resolution Sure

Solution 2131470.1 : SDM 7.5 Cluster Configuration : Adding a New Cluster gives Error "No route to Host"

Applies to:

Acme Packet 6300 - Version S-Cz7.2.0 and later
Oracle Communications Session Delivery Manager - Version 7.x and later
Information in this document applies to any platform.

Symptoms

Customer used to have SDM 7.5 running on a stand-alone server, and they were trying the cluster configuration with 2 nodes:

• ServerA with IP xx.xx.xx.147 as the master node
• ServerB with IP xx.xx.xx.148 as the backup node

For this they ran the cluster config on both node:
- stop SDM service
- ran setup.sh with Custom config
- Choose “6 - OC SDM cluster management”.
- Finish the config
- Start the nnc service

On both node, they has a no route to host error, which means being in the same subnet as well, they were unable to setup a cluster communication.

If we see ServerA DbService.log file

2016-03-08 10:42:40,303 INFO [com.acmepacket.ems.server.services.database.DBErrorHandler] - Method: [ReplicatedXMLDatabaseManagerImpl] Thread: [Thread-254:925] Msg:[connecting to site xx.xx.xx.148:9000: No route to host]
2016-03-08 10:43:10,506 INFO [com.acmepacket.ems.server.services.database.DBErrorHandler] - Method: [ReplicatedXMLDatabaseManagerImpl] Thread: [Thread-255:929] Msg:[connecting to site xx.xx.xx.148:9000: No route to host]

We had the same error on serverB.

Ping between both these nodes was working fine. They were on same customer LAN. Here is the route output:

[nncentral@serverA ~]$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default xx.xx.xx.130 0.0.0.0 UG 0 0 0 eth0
[…]
xx.xx.xx.128 * 255.255.255.224 U 0 0 0 eth0

Cause

There was connectivity issue among the cluster nodes. The TCP socket error "NoRouteToHostException" shows that this exception either was caused by the firewall or an intermediate router is down.

2016-02-09 14:14:51,024 ERROR [com.acmepacket.ems.server.services.database.Document] - Method: [fixConfigFileWithSanitizer] Thread: [RMI TCP Connection(43149)-xx.xx.xx.147:609795] Msg:[Exception thrown; Reason = ]

2016-02-09 14:15:45,107 [com.acmepacket.ems.server.services.database.Document] - Method: [fixConfigFileWithSanitizer] Thread: [RMI TCP Connection(43149)-xx.xx.xx.147:609795] Msg:[Exception thrown; Reason = ]

2016-02-09 14:17:53,354 ERROR [com.acmepacket.ems.server.services.database.Document] - Method: [fixConfigFileWithSanitizer] Thread: [RMI TCP Connection(43160)-xx.xx.xx.147:609887] Msg:[Exception thrown; Reason = ]

2016-03-08 10:40:39,490 INFO [com.acmepacket.ems.server.services.database.DBErrorHandler] - Method: [ReplicatedXMLDatabaseManagerImpl] Thread: [Thread-250:908] Msg:[connecting to site xx.xx.xx.148:9000: No route to host]

The cause of this issue could be:

1) If customers have configured iptables to allow or reject traffic from specific IP addresses or ports.

2) No firewall should be deployed between the cluster members. The main reason is due to the dynamic assignment of RMI port, iptables has to be disabled on the servers.

To see if iptables are configured, run command "iptables -L -n -v" from root user.

Solution

Customer disabled iptables after which the error disappeared and they were able to setup the cluster-config.

Goal

SDM cluster-config error: no route to host

Attachments

This solution has no attachment