How to Clear DSR Alarm ID 31215 "Process Resources Exceeded" for Instance "re.portmap" Warning of Heap Growth

Asset ID:	1-71-2030060.1
Update Date:	2018-01-16
Keywords:

Solution Type Technical Instruction Sure

Solution 2030060.1 : How to Clear DSR Alarm ID 31215 "Process Resources Exceeded" for Instance "re.portmap" Warning of Heap Growth

Applies to:

Oracle Communications Diameter Signaling Router (DSR) - Version DSR 5.0 and later
Information in this document applies to any platform.

Goal

Resolve Minor Event 31215 alarm 'Process Resources Exceeded' when raised against the process re.portmap for heap growth.

Solution

This is an alarm indicating that a certain process named in the alarm itself--in this case re.portmap--is consuming more resources--in this instance 'heap' or memory--than is expected based on an alarm threshold specified in the code for this process. The workaround resolution for this condition is to restart the process to free the resources. For this specific process -- re.portmap -- restarting the process will have no effect on operation or traffic.

NOTE: This resolution may not be applicable to all processes for which Minor Alarm ID 31215 may appear; this document is applicable for re.portmap

Sample Alarm:

             TIMESTAMP: YYYY-MM-DD HH:MM:SS.xxx UTC
NETWORK_ELEMENT: typically_soam_server
                 SERVER: server_in_alarm
                SEQ_NUM: ######
      EVENT_NUMBER: 31215
             SEVERITY: MINOR
           PRODUCT: Platform
               PROCESS: ProcWatch
                TYPE: SW
              INSTANCE: re.portmap
                  NAME: Process Resources Exceeded
              DESCR: A process is consuming excessive system resources
               ERR_INFO:
   GN_WARNHEAP/WRN heap growth detected (size=#######K) for cmd=xds pid=#PID#[ProcWatchMain.cxx:627]
^^ threshold type=minute [ProcWatchMain.cxx:630]
^^ heap growth count=## [ProcWatchMain.cxx:631]
^^ heap growth threshold=# [ProcWatchMain.cxx:632
       <SECS:,USECS:,CISECS:,CIUSECS:,ID: all removed for brevity>

Resolution Steps:

Log in to server (SERVER named in alarm) at the command line as root user.
Verify alarm is present:
[root@SERVER ~]# ra.stat -w | grep -i re.portmap
Sample output:
+ iqt -z -fseverity,as=sev -fhits,as=ht,wid=2 -ftimeStamp,fmt=Time.M -feventNumber,as=event,wid=24,trunc,left -finstance,wid=12,left,over -ferrInfo,over,nonl RecentAlarmEv.0 by severity,timeStamp
-* # HH:MM:SS.xxx Process Resources Exceed re.portmap      GN_WARNHEAP/WRN heap growth detected (size=#######K) for cmd=re.portmap pid=#PID# [ProcWatchMain.cxx:627]   ^^ threshold type=minute
ProcWatchMain.cxx:630]   ^^ heap growth count=## [ProcWatchMain.cxx:631]   ^^ heap growth threshold=# [ProcWatchMain.cxx:632]   ^^ [33545:ProcWatchMain.cxx:637]
Determine the process ID (PID):
[root@SERVER ~]# pl | grep -i re.portmap
Sample output:
A #PID# re.portmap Up MM/DD HH:MM:SS # re.portmap -c100
Restart the process:
[root@SERVER ~]# pm.kill re.portmap
Verify the process ID (PID) and process timestamp has changed:
[root@SERVER ~]# pl | grep -i re.portmap
Sample output:
A PID## re.portmap              Up   MM/DD HH:MM:SS # re.portmap -c100
Alarm should clear within 5 minutes. To verify via command line:
[root@SERVER ~]# ra.stat -w | grep -i re.portmap
Sample output:
+ iqt -z -fseverity,as=sev -fhits,as=ht,wid=2 -ftimeStamp,fmt=Time.M -feventNumber,as=event,wid=24,trunc,left -finstance,wid=12,left,over -ferrInfo,over,nonl RecentAlarmEv.0 by severity,timeStamp

<should be no output>

These steps should clear the condition.

Attachments

This solution has no attachment