Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-72-2214601.1
Update Date:2017-11-12
Keywords:

Solution Type  Problem Resolution Sure

Solution  2214601.1 :   ProTrace/Troubleshooting Shows "Could not load the PDUs" Instead of Summary and Full Decoding Because of "Port already in use"  


Related Items
  • Oracle Communications Performance Intelligence Center (PIC) Software
  •  
Related Categories
  • PLA-Support>Sun Systems>CommsGBU>Global Signaling Solutions>SN-SND: Tekelec PIC
  •  




In this Document
Symptoms
Changes
Cause
Solution
 Verify if patch is installed
References


Created from <SR 3-13837059291>

Applies to:

Oracle Communications Performance Intelligence Center (PIC) Software - Version 10.2.0 and later
Information in this document applies to any platform.

Symptoms

Protrace full decoding is not displayed and "Coult not load the PDUs" is displayed instead.

/var/TKLC/log/ixp/DsapiJavaServer.log contains (from server where store is distributed):

12/13/16 9:30:18 AM DSAPI: Unable to get Dsapi remoteException: java.rmi.server.ExportException: Port already in use: 5055; nested exception is:
java.net.BindException: Address already in use 

Changes

 

Cause

Dsapi process cannot bind on port that is already used.

Solution

Latest mediation patch must be installed (more information in 2118743.1).
After applying workaround, any incremental upgrade will revert the workaround.

On all servers part of the subsystem:

  1. Log on IXP server as cfguser:
    su - cfguser
  2. Change directory:
    cd /opt/TKLCixp/prod/bin/
  3. Stop dsapi process:
    pm.set off dsapi
  4. Steps for ProlibServer:
    1. Rename file:
      mv ProlibServer ProlibServer_exe
    2. Edit new ProlibServer file:
      vim ProlibServer
      With following content (point is important):
      #!/bin/sh
      . PortCloser.sh
  5. Create new file PortCloser.sh:
    vim PortCloser.sh
    With following content:
    #!/bin/sh
     function close_port
    {
    _PID_FD=( `lsof -F f -i TCP:$1 -s TCP:LISTEN|cut -c 2-|grep -A1 $$` )
    if [ ${#_PID_FD[*]} -eq 2 ]; then
    gdb -p ${_PID_FD[0]} -batch-silent -ex "call close(${_PID_FD[1]})"
    fi
    }
    close_port 5055
    close_port 5031
    exec `basename $0`_exe
  6. Step for WiresharkServer:
    1. Rename WiresharkServer:
      mv WiresharkServer WiresharkServer_exe
    2. Create a copy of script:
      cp ProlibServer WiresharkServer
  7. Change rights:
    chmod u+x ProlibServer WiresharkServer PortCloser.sh
  8. Start dsapi process:
    pm.set on dsapi
  9. Verify with pm.getprocs that dsapi does not restart in loop
  10. After few seconds, verify ports are opened properly
    lsof -i TCP:5055 -i TCP:5031 -s TCP:LISTEN
    Expected output:
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    java 5113 cfguser 17u IPv6 21423 0t0 TCP *:unot (LISTEN)
    java 5113 cfguser 11u IPv6 21415 0t0 TCP *:5031 (LISTEN)
    If you see more lines, it means that one step was missed during this procedure.

Full decoding will be properly displayed and issue will not reoccur.

Verify if patch is installed

  1. TKLCixp package must be at least 10.2.0.0.0-25.5.0 (For latest information on 10.2.0, check 2118743.2):
    # rpm -q TKLCixp
    TKLCixp-10.2.0.0.0-25.5.0.x86_64
  2. Run following command:
    lsof -i TCP:5055 -i TCP:5031 -s TCP:LISTEN
    Expected output (one process per port):
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    java 5113 cfguser 17u IPv6 21423 0t0 TCP *:unot (LISTEN)
    java 5113 cfguser 11u IPv6 21415 0t0 TCP *:5031 (LISTEN)
    If you see more lines, it means that one step was missed during this procedure.
  3. Verify file type and presence:
    cd /opt/TKLCixp/prod/bin/;file PortCloser.sh WiresharkServer WiresharkServer_exe ProlibServer ProlibServer_exe
    Expected output:
    PortCloser.sh: POSIX shell script text executable
    WiresharkServer: POSIX shell script text executable
    WiresharkServer_exe: ELF 64-bit LSB executable (...)
    ProlibServer: POSIX shell script text executable
    ProlibServer_exe: ELF 64-bit LSB executable (...)
    If output is different modifications were not properly implemented. Verify if any step was missed or recover file from another server of the subsystem

Scripted version

  1. Download and copy attached script on one IXP server as cfguser.
  2. Copy on all server but current:
    for i in `iqt -zph -f_name DaqServer| grep -v \`hostname\``; do echo $i;scp install_dsapi_fix.pl cfguser@$i:/home/cfguser/install_dsapi_fix.pl; done
  3. Install it on all servers:
    for i in `iqt -zph -f_name DaqServer`; do echo $i;ssh $i "./install_dsapi_fix.pl"; done
  4. Restart Dsapi on all servers:
    for i in `iqt -zph -f_name DaqServer`; do echo $i;ssh $i "pm.set off dsapi; sleep 3; pm.set on dsapi"; done

 

References

<BUG:23624848> - FULL DECODING NOT DISPLAYED IN PROTRACE
<NOTE:2118743.1> - Which Performance Intelligence Center (PIC) 10.2.0 Patches to Update and How to Install Them?

Attachments
This solution has no attachment
  Copyright © 2018 Oracle, Inc.  All rights reserved.
 Feedback