How To Clear Thread Watchdog Failure Alarm ID 31003 Bouncing Due Misconfigured DSCP

Asset ID:	1-71-2310094.1
Update Date:	2017-12-08
Keywords:

Solution Type Technical Instruction Sure

Solution 2310094.1 : How To Clear Thread Watchdog Failure Alarm ID 31003 Bouncing Due Misconfigured DSCP

Applies to:

Oracle Communications Diameter Signaling Router (DSR) - Version DSR 7.1.2 and later
Information in this document applies to any platform.

Goal

Alarms 31003 appearing on C Level Servers [MP/IPFE]

Following is the sample alarm.

TIMESTAMP: 2017-09-18 11:17:07.176 MSK
NETWORK_ELEMENT: SOAM_ABC
  SERVER: ABC-MP4
  SEQ_NUM: 59204
  EVENT_NUMBER: 31003
  SEVERITY: MINOR
  PROCESS: apwSoapServer
  TYPE: SW
  INSTANCE: apwSoapServer
  NAME: Thread Watchdog Failure
  DESCRIPTION: Thread watchdog timed out
  ERR_INFO:
  GN_EXPIRED/WRN Thread watchdog expired for thread ID 5973 (AuditManager) [SigCtl.cxx:194]
  ^^ [5763:SigCtl.cxx:197]

  SECS: 1505722627
  USECS: 176000
  CISECS: 1505722627
  CIUSECS: 224000
  ID: 0

Solution

Before applying this Procedure kindly check the following

Error logs apwSoapServer.log file on MP/IPFE Server, Following is error is displayed.
#tail -f apwSoapServer.log

2017-09-19 10:40:33.224 [139968995403520] ERROR log.audit <> - DscpAudit::auditDscpByIntfInsert: iptablesAdm failed to configure resource: ABC-IPFE2/ABC-IPFE2//24/tcp/ipv4 [audit/DscpAudit.C:452]
2017-09-19 10:40:43.035 [139968995403520] ERROR log.audit <> - DscpAudit::auditDscpByIntfInsert: iptablesAdm failed to configure resource: ABC-IPFE2/ABC-IPFE2//24/sctp/ipv4 [audit/DscpAudit.C:452
Check the DSCP Configuration, Login on Active NOAM GUI Navigate to “Main Menu: Configuration -> DSCP -> Interface DSCP”, and select “ABC-IPFE2” tab. You can see the re the “empty” interface records there , Below are the empty records (four records in my case) where we can see interface is blank.
Id: 956
scope: ABC_IPFE2_SG
scopeType: SG
server: ABC-IPFE2
interface: => Blank
dscp: 24
protocol: TCP

To Solve the Issue.

Delete all the Configuration which is showing interface as empty.Select the interface > Click Delete Button at Bottom of the Page > Click OK to delete. Recreate the DSCP Interface correctly.

Check the Error messages in "tail -f apwSoapServer.log
Once this is done, Issue should be resolved.

References

<BUG:26845063> - DSCPBYINTF TABLE BOGUS DEVICE ENTRIES CAUSE 31003 THREAD WATCHDOG ALARM
<BUG:26845090> - DSCPBYINTF TABLE BOGUS DEVICE ENTRIES CAUSE 31003 THREAD WATCHDOG ALARM

Attachments

This solution has no attachment