Oracle Communications Performance Intelligence Center (PIC) Software
 
Oracle Communications Diameter Signaling Router (DSR)
 

PLA-Support>Sun Systems>CommsGBU>Global Signaling Solutions>SN-SND: Tekelec PIC
 

In this Document

Symptoms

Cause

Solution

Definitive solution

Workaround

Restore

Prevent

References

Created from <SR 3-10597021831>

Applies to:

Symptoms

Following symptoms can happen:

• xDR session stopped with following log in IxpStore or IxpOperate:
  1214:171141.562 DTS2 OpenConnection(): Connecting to IP address ixphostname failed, port is 2222 and stream name is 'streamname' [19203/DtsConsumer.cpp:183]
  1214:171141.562 DTS2 OpenStream(): OpenConnection failure with error message '-10' on stream name 'streamname' [19203/DtsConsumer.cpp:568]
• IXP server is rebooting automatically. hplog -v command output shows Automatic Server Recovery (ASR) has been triggered.
• top shows high load (over 100)
• IxpForward use maximum allocatable memory on a 32 bits operating system (around 3 Gigabytes)
• IxpForward log contains "Cannot allocate memory" messages.

Cause

IxpForward is allocating memory for each connection attempts. After some time, process can reach operating system limit. 

Solution

Definitive solution

It is highly recommended to upgrade to PIC 10.1.

Workaround

Restore

If IxpForward is currently logging "Cannot allocate memory" messages, it can be restarted.

Prevent

The only workaround consist of checking if there are still repeated connection attempts received by IxpForward process.

In normal operation, no new connection attempts shall be made.

Log message looks like:

If this kind of messages are regularly written, it must be investigated following procedure below:

1. Collect IxpForward logs from all IXP base or PDU servers still having connection attempts:
2. All files can be parsed with command below to know streams names and number of occurences:
   for i in `ls *.log`; do grep -oP 'Connection on [a-zA-Z_0-9]* at' $i | cut -c15- | rev|cut -c3- | rev | sort | uniq -c > $i.out; done ; grep '' *.out
3. Analyze each connection attempts. Use show_stream_info.pl and show_process_info.pl script to find more information on streams and DFP.
   Connection attempts can be due to:
   • empty stream (no data)
   • crashing process

References

This solution has no attachment