Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-72-2376687.1
Update Date:2018-05-29
Keywords:

Solution Type  Problem Resolution Sure

Solution  2376687.1 :   SBC Not Responding to Inbound INVITE and REGISTER. TCP Connection Requests Rejected by SBC  


Related Items
  • Acme Packet 6300
  •  
Related Categories
  • PLA-Support>Sun Systems>CommsGBU>Session Delivery Network>SN-SND: Acme Service Provider
  •  




In this Document
Symptoms
Changes
Cause
Solution
References


Created from <SR 3-16851728161>

Applies to:

Acme Packet 6300 - Version S-Cz7.2.0 to S-Cz7.4.0 [Release S-Cz7.0]
Acme Packet OS

Symptoms

Phones or Endpoints unable to register and make calls.

SBC rejecting incoming INVITE and REGISTER requests with 480 or 503 response.

TCP connection attempts are being rejected by SBC with RST,ACK response.

Changes

A network event appears to have been the catalyst to the problem, causing load balancing among multiple SBCs to fail.

Rather than balance registration attempts among the many SBCs, registration attempts spiked on one SBC.

Cause

With a notable increase in registration (or SIP) traffic, the SBC transport threads begin to queue these inbound messages.

Once the transport queues start backing up and exceed the queue size, the SBC will eventually start to drop packets, resulting in re-transmissions and further new connection attempts towards the SBC.

Depending upon the version of code present on the SBC, an issue can be uncovered where the SBC code erroneously utilizes the wrong file descriptor.

If this file descriptor happens to related to the TCP listening socket associated with the sip-interface, the listening socket can be reset.

The result would prevent the SBC from allowing new TCP connection attempts.

 

There are several CLI commands that will show this potential problem which could cause the SBC to attempt to close the TCP listening socket.

Three of the key indicators are seen below.

1)  show ip connections

Verify that the TCP socket for the sip-interface(s) is in TCPS_LISTEN state for all foreign addresses. 

If the socket is in any other state or if NOT present in the output, a problem exists.

 &nbsp;
ATCP Active Internet connections (including servers)
PCB Socket Recv-Q Send-Q Local Address Foreign Address (state)
-------- -------- ------ ------ --------------------- --------------------- -------
0x31e34dc0 0x2f240d00 0 0 1.20.1.2.5061 0.0.0.0.0 TCPS_LISTEN
 &nbsp;

 

2)  show buffers

If the "number of times failed to find space" contains a large and increasing value, the problem may exist.

show buffers
type number
--------- ------
FREE : 1364
TOTAL : 64000
number of mbufs: 32000
number of times failed to find space: 19940690

 

3)  show queues

If Events Pending and Events Dropped are increasing, the problem may exist.

<<< show queues
01:12:58-112
Thread Queue Statistics -- Period -- -------- Lifetime --------
Active High Total Total PerMax High
Transport Queues 2 2 0 2 2 2
Waiting Threads 2 2 327226 7150617 300910 2
Events Pending 0 31 590772 13453001 1494K 12990
Events Dropped 0 0 0 706979 495957 414

 

Some errors seen in log.atcpd, found in the /opt/logs/ directory, which indicate a problem are seen below.

log.atcpd:Jan 1 00:00:00.111 [SERVICE] (2) 5000 TransportQueue commands queued; events=1218179 queue size=5000/-1 maxQ=6001 dropped=165364 wait=423/500
log.atcpd:Jan 1 00:00:00.111 [SERVICE] (2) 4000 TransportQueue commands queued; events=1243013 queue size=4000/-1 maxQ=5001 dropped=165364 wait=321/500

 

Solution

A defect was created and fixed to correct this problem.  An upgrade to a release that contains the bug fix is the ultimate and final resolution for this problem.

Fix included in the following releases:

SCZ730m3p2 and later

SCZ740M1 and later

SCZ8.0.0

The problem can exist in releases earlier than those noted above and was initially found within the SCZ730 code stream.

 

Code changes have been implemented to correct the conditions under which the SBC resets the TCP listen socket.

The ultimate fix is to upgrade to a release that contains this fix.

If necessary, a Service-Request can be opened with Oracle Support in order to determine the correct upgrade path.

 

Temporary solutions can include:

1)  Reboot of SBC, coupled with throttling of inbound traffic.  Throttling measures can differ for various environments.

2)  Modify TCP listening port. 

Example, if 5061 is not in TCPS_LISTEN state, the sip-port configuration can be modified to 5065 (or other), followed by save and activate. Then the configuration would to be modified back to use the correct sip-port of 5061, followed by a second save and activate. This would cause the SBC to resume proper handling of these packets. 

 

 

 

References

<BUG:26316830> - USM STOPS ACCEPTING TCP CONNECTIONS
<BUG:27550421> - TLID CRASH ON VGW SBC SOUTHLAKE 01.
<BUG:26381224> - TRUNK: CHANGE ATCPD HANDLE ALLOCATOR TO FIFO
<BUG:27535158> - VGW - INCREASE IN 480 UNAVAILABLE

Attachments
This solution has no attachment
  Copyright © 2018 Oracle, Inc.  All rights reserved.
 Feedback