Asset ID: |
1-72-2214601.1 |
Update Date: | 2017-11-12 |
Keywords: | |
Solution Type
Problem Resolution Sure
Solution
2214601.1
:
ProTrace/Troubleshooting Shows "Could not load the PDUs" Instead of Summary and Full Decoding Because of "Port already in use"
Related Items |
- Oracle Communications Performance Intelligence Center (PIC) Software
|
Related Categories |
- PLA-Support>Sun Systems>CommsGBU>Global Signaling Solutions>SN-SND: Tekelec PIC
|
In this Document
Created from <SR 3-13837059291>
Applies to:
Oracle Communications Performance Intelligence Center (PIC) Software - Version 10.2.0 and later
Information in this document applies to any platform.
Symptoms
Protrace full decoding is not displayed and "Coult not load the PDUs" is displayed instead.
/var/TKLC/log/ixp/DsapiJavaServer.log contains (from server where store is distributed):
12/13/16 9:30:18 AM DSAPI: Unable to get Dsapi remoteException: java.rmi.server.ExportException: Port already in use: 5055; nested exception is:
java.net.BindException: Address already in use
Changes
Cause
Dsapi process cannot bind on port that is already used.
Solution
Latest mediation patch must be installed (more information in
2118743.1).
After applying workaround, any incremental upgrade will revert the workaround.
On all servers part of the subsystem:
- Log on IXP server as cfguser:
su - cfguser
- Change directory:
cd /opt/TKLCixp/prod/bin/
- Stop dsapi process:
pm.set off dsapi
- Steps for ProlibServer:
- Rename file:
mv ProlibServer ProlibServer_exe
- Edit new ProlibServer file:
vim ProlibServer
With following content (point is important):
#!/bin/sh
. PortCloser.sh
- Create new file PortCloser.sh:
vim PortCloser.sh
With following content:
#!/bin/sh
function close_port
{
_PID_FD=( `lsof -F f -i TCP:$1 -s TCP:LISTEN|cut -c 2-|grep -A1 $$` )
if [ ${#_PID_FD[*]} -eq 2 ]; then
gdb -p ${_PID_FD[0]} -batch-silent -ex "call close(${_PID_FD[1]})"
fi
}
close_port 5055
close_port 5031
exec `basename $0`_exe
- Step for WiresharkServer:
- Rename WiresharkServer:
mv WiresharkServer WiresharkServer_exe
- Create a copy of script:
cp ProlibServer WiresharkServer
- Change rights:
chmod u+x ProlibServer WiresharkServer PortCloser.sh
- Start dsapi process:
pm.set on dsapi
- Verify with pm.getprocs that dsapi does not restart in loop
- After few seconds, verify ports are opened properly
lsof -i TCP:5055 -i TCP:5031 -s TCP:LISTEN
Expected output:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 5113 cfguser 17u IPv6 21423 0t0 TCP *:unot (LISTEN)
java 5113 cfguser 11u IPv6 21415 0t0 TCP *:5031 (LISTEN)
If you see more lines, it means that one step was missed during this procedure.
Full decoding will be properly displayed and issue will not reoccur.
Verify if patch is installed
- TKLCixp package must be at least 10.2.0.0.0-25.5.0 (For latest information on 10.2.0, check 2118743.2):
# rpm -q TKLCixp
TKLCixp-10.2.0.0.0-25.5.0.x86_64
- Run following command:
lsof -i TCP:5055 -i TCP:5031 -s TCP:LISTEN
Expected output (one process per port):
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
java 5113 cfguser 17u IPv6 21423 0t0 TCP *:unot (LISTEN)
java 5113 cfguser 11u IPv6 21415 0t0 TCP *:5031 (LISTEN)
If you see more lines, it means that one step was missed during this procedure.
- Verify file type and presence:
cd /opt/TKLCixp/prod/bin/;file PortCloser.sh WiresharkServer WiresharkServer_exe ProlibServer ProlibServer_exe
Expected output:
PortCloser.sh: POSIX shell script text executable
WiresharkServer: POSIX shell script text executable
WiresharkServer_exe: ELF 64-bit LSB executable (...)
ProlibServer: POSIX shell script text executable
ProlibServer_exe: ELF 64-bit LSB executable (...)
If output is different modifications were not properly implemented. Verify if any step was missed or recover file from another server of the subsystem
Scripted version
- Download and copy attached script on one IXP server as cfguser.
- Copy on all server but current:
for i in `iqt -zph -f_name DaqServer| grep -v \`hostname\``; do echo $i;scp install_dsapi_fix.pl cfguser@$i:/home/cfguser/install_dsapi_fix.pl; done
- Install it on all servers:
for i in `iqt -zph -f_name DaqServer`; do echo $i;ssh $i "./install_dsapi_fix.pl"; done
- Restart Dsapi on all servers:
for i in `iqt -zph -f_name DaqServer`; do echo $i;ssh $i "pm.set off dsapi; sleep 3; pm.set on dsapi"; done
References
<BUG:23624848> - FULL DECODING NOT DISPLAYED IN PROTRACE
<NOTE:2118743.1> - Which Performance Intelligence Center (PIC) 10.2.0 Patches to Update and How to Install Them?
Attachments
This solution has no attachment