NSP GUI Unreachable with Message "Failure of APACHE bridge: No backend server available for connection timed out after 60 seconds or idempotent set to OFF"

Asset ID:	1-72-2264193.1
Update Date:	2017-05-10
Keywords:

Solution Type Problem Resolution Sure

Solution 2264193.1 : NSP GUI Unreachable with Message "Failure of APACHE bridge: No backend server available for connection timed out after 60 seconds or idempotent set to OFF"

Applies to:

Oracle Communications Performance Intelligence Center (PIC) Software - Version 9.0.3 and later
Information in this document applies to any platform.
The syscheck shows that /usr directory is full.

Symptoms

NSP GUI is not reachable thru Web browser, showing error message "Failure of APACHE bridge: No backend server available for connection timed out after 60 seconds or idempotent set to OFF."

Changes

The /usr folder was full and several useless files have been removed from there to have available space in this directory.

Cause

Some essential files for Weblogic were corrupted because of /usr folder was full.

usr/TKLC/nsp/bea/user_projects/domain/tekelec/servers/nspadmin/data/ldap/conf/replicas.prop
usr/TKLC/nsp/bea/user_projects/domain/tekelec/servers/nsp1a/data/ldap/conf/replicas.prop
usr/TKLC/nsp/bea/user_projects/domain/tekelec/servers/nsp1b/data/ldap/conf/replicas.prop

Verify whether these are 0 byte files, so not usable by the system, preventing Weblogic to start correctly.

nspadmin.log file shows the following error:

####<May 2, 2017 5:01:10 PM EEST> <Critical> <WebLogicServer> <NSP1> <nspadmin> <Main Thread> <<WLS Kernel>> <> <> <1493733670534> <BEA-000386> <Server subsystem failed. Reason: java.lang.NumberFormatException: null
java.lang.NumberFormatException: null
at java.lang.Integer.parseInt(Integer.java:417)
at java.lang.Integer.parseInt(Integer.java:499)
at weblogic.ldap.EmbeddedLDAP.validateVDEDirectories(EmbeddedLDAP.java:1104)
at weblogic.ldap.EmbeddedLDAP.start(EmbeddedLDAP.java:242)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)

Solution

As the error (displayed as "Failure of APACHE bridge") is related to corrupted files, these files must be generated again (automatically by Weblogic).
To allow this operation, current files replicas.prop must be removed from the various directories.

Log on to NSP weblogic primary server as root and do:

rm -f usr/TKLC/nsp/bea/user_projects/domain/tekelec/servers/nspadmin/data/ldap/conf/replicas.prop
rm -f usr/TKLC/nsp/bea/user_projects/domain/tekelec/servers/nsp1a/data/ldap/conf/replicas.prop
rm -f usr/TKLC/nsp/bea/user_projects/domain/tekelec/servers/nsp1b/data/ldap/conf/replicas.prop

Then, nsp must be restarted:

service nspservice stop
service nspservice start

Check the files that should have been regenerated. In nspadmin directory it should look like:

#Generated property file
#Fri May 05 12:17:26 EEST 2017
replica.0.port=7001
replica.0.consumerid=nsp1a
replica.0.masterurl=ldap\://10.4.250.53\:8001/
replica.0.secure=0
replica.1.port=7003
replica.1.secure=0
replica.1.consumerid=nsp1b
replica.0.binddn=cn\=Admin
replica.1.masterurl=ldap\://10.4.250.53\:8001/
replica.1.hostname=10.4.250.53
replica.num=2
replica.0.name=nsp1a
replica.1.name=nsp1b
replica.1.binddn=cn\=Admin
replica.0.base=dc\=tekelec
replica.0.hostname=10.4.250.53
replica.1.masterid=nspadmin
replica.0.masterid=nspadmin
replica.1.base=dc\=tekelec

In nsp1a and nsp1b directories, it looks like:

replica.num=0

Verify whether the NSP GUI can be accessed.

Attachments

This solution has no attachment