Oracle Communications Subscriber Data Management (SDM)
 

PLA-Support>Sun Systems>CommsGBU>Broadband Network Solutions>SN-SND: Tekelec SDM
 

In this Document

Purpose

Troubleshooting Steps

Applies to:

Purpose

 The following document provides steps needed to resolve provisioning failures where the system has exceeded the maximum number of SOAP.

Oracle has confirmed that bug 20521266 is present in SPR SDM 9.3.1 software. For systems running with software prior to patch 7, the SPR is unable to calculate the number of SOAP and SOAP-DAS connections properly and SPR goes into error condition where the internal number of connection exceeded beyond allowed limit, which is 42.

Per product design, maximum allowed simultaneous requests could go up to 42(7 end points(max) X 6 requests(max)/endpoint) between provisioning client and XDS(SOAP Server) and if these requests are completed within 1 seconds then more than 42 requests could be allowed. This cannot be increased to accommodate more.

It's important to note that Oracle SDM 9.3.1 Patch 7 provides corrective content to resolve this failure, however those customers below patch 7 are prone for this failure and this document covers how to restore the provisioning failure.

Troubleshooting Steps

 If you are experiencing a Provisioning failure on SDM 9.3.1. Confirm the software level patch level you are on by executing the following commands:

# BlueVersion

# bluepatch status

Based on output of data:

• If your System is 9.3.1 are on patches greater than or equal to 7, this procedure does not apply and you should immediately open an SR with Support so the issue could be further investigated.
• If your System is not 9.3.1, then this procedure does not apply and you should immediately open an SR with Support so the issue could be further investigated.
• If your System is 9.3.1 are on patches less than 7, this procedure does apply and you should follow the below steps.

Validation

1. Unable to execute SOAP provisioning and receiving errors "Communication problem towards NE" or "Connection lost with geographic database"

2. Check the internal session currently established between SOAP and DAS server , please execute below command:

# netstat -anp | grep -i xmlDataServer | egrep '62001' | wc -l

SESSION: Connected to 169.254.1.20, port: 62001, ip: 10.136.58.19, user: OSS, ID: 3
SESSION: Connected to 169.254.1.20, port: 62001, ip: 10.136.58.36, user: OSS, ID: 2
SESSION: Connected to 169.254.1.20, port: 62001, ip: 10.136.58.27, user: OSS, ID: 3
SESSION: Connected to 169.254.1.20, port: 62001, ip: 10.136.58.36, user: OSS, ID: 4

The below error log is found in /blue/var/log/WebCI.out file:

Exception too many connections openned!!!
com.blueslice.exception.ManyConnectionException: Error too many SOAP connections opened!!!
at com.blueslice.service.Session.LoginSession(Session.java:258)
at com.blueslice.service.Session.authenticate(Session.java:115)
at com.blueslice.service.SessionInfo.getSession(SessionInfo.java:49)
at com.blueslice.service.PoolSessions.getSession(PoolSessions.java:33)
at com.blueslice.webservice.MessageService.processTransaction(MessageService.java:115)
at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.axis.providers.java.MsgProvider.processMessage(MsgProvider.java:126)
at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)

Corrective Action Plan

If the above is not seen, then open SR with Support for immediate assistance.

If the above Exception error is seen the use the following to restore provisioning.

Stop WebCI :
==========

# WebCI stop

Check the number of connection :
==========================

# netstat -anp | grep <provisioning IP>

Wait until above count become 0 or less than 5

Start WebCI process:
=================

# WebCI start

Data Collection

The following data collection can be of value to Oracle for deeper investigation:

1. Log/trace files
2. Get the ss-a output
3. Output of: lsof | grep dataAcc
4. Output of lsof | grep xmlData

End of Procedure.

If you are not on Patch 7, then please open SR for Support to apply all patches to your system. Current patch level is 16.

This solution has no attachment