How to understand and handle UDR Alarm Event 31215 - Process Resources Exceeded Alarm

Asset ID:	1-71-2182224.1
Update Date:	2018-05-21
Keywords:

Solution Type Technical Instruction Sure

Solution 2182224.1 : How to understand and handle UDR Alarm Event 31215 - Process Resources Exceeded Alarm

Applies to:

Oracle Communications User Data Repository - Version UDR 10.0 and later
Tekelec

Goal

The following information provides information on UDR Alarm Event 31215 - Process Resources Exceeded

DESCR: A process is consuming excessive system resources

GN_WARNHEAP/WRN heap growth detected (size=1606408K)

Solution

From the Alarms, KPIs, and Measurements Reference Document

31215 - Process resources exceeded
Alarm Group: SW
Description: A process is consuming excessive system resources
Severity: Minor
Instance: May include AlarmLocation, AlarmId, AlarmState, AlarmSeverity, and bindVarNamesValueStr
HA Score: Normal
Auto Clear Seconds: 14400
OID: comcolProcResourcesExceededFaultNotify

The manual provides instructions to customers to "Contact My Oracle Support (MOS)".

The following information is provided help obtain additional information about the generation of the alarm.

A. What is the maximum heap size assigned to UDR process?
The UDR Process is configured to take up to 23GB heap size. However, the typical heap size usage of UDR process to 1.6GB.

B. How do you check the current heap size assigned to UDR process?
Log into server command line and use the following command to check the current heap size

# pmapsum $(pidof udr)

Also, heap size changes is monitored in Alarm History as INFO, MINOR, MAJOR, and CRITICAL Alarms.
Value indicated under data corresponds to heap usage of the process. Unit for the value is MB.

Example output:

[root@pc9000712-mp ~]# pmapsum $(pidof udr)
text data stack shOth total(MB) shMem prog/pid
15.4 1,434.5 0.0 0.1 1,450.1 9.0 udr 25767

C. What is the criteria size to raise below alarm? How do you see the configuration?
Criteria to raise the alarm is the same as described below. This configuration cannot be viewed by GUI.

The default settings are as follows:

For INFO event:
HeapSize >= 256M and one of the following:
• 8 consecutive growths detected measured over 15-minute intervals
• 5 consecutive growths detected measured over 2-hour intervals
• 4 consecutive growths detected measured in 1-day intervals

For WARNING alarm:
HeapSize >= 1G and one of the following:
• 8 consecutive growths detected measured over 15-minute intervals
• 5 consecutive growths detected measured over 2-hour intervals
• 4 consecutive growths detected measured in 1-day intervals

D. What are the factors that will make the heap size change to critical?
Continuously growing heap size would be critical as it would mean that the memory is leaking.

Example:

INSTANCE NAME DESCRIPTION ERR_INFO
udr Process Resources Exceeded A process is consuming excessive system resources Alarm auto clear
udr Process Resources Exceeded A process is consuming excessive system resources GN_WARNHEAP/WRN heap growth detected (size=1606408K) for cmd=udr pid=14318 [ProcWatchMain.cxx:626] ^^ threshold type=hour [ProcWatchMain.cxx:629] ^^ heap growth count=20 [ProcWatchMain.cxx:630] ^^ heap growth threshold=20 [ProcWatchMain.cxx:631] ^^ [10949:ProcWatchMain.cxx:636]

E. What to do when this condition occurs?
The following information should be collected from the Server and attached to SR for Oracle investigation.

Export “Alarm History” into csv format.
The alarm events should be for 5 days.
1. Execute Filter on Alarms & Events – View History.
2. Select the “Export” radio button.
3. Select “Ok” on the Schedule Event Data Export
4. Select “Tasks” to view drop down
a. New Task will show completed at 100%.
b. Left click on “Events*”. Execute steps to save file to desktop.
5. Attach file to SR.

Attachments

This solution has no attachment