Oracle Communications Diameter Signaling Router (DSR)
 

PLA-Support>Sun Systems>CommsGBU>Global Signaling Solutions>SN-SND: Tekelec DSR
 

In this Document

Symptoms

Changes

Cause

Solution

Created from <SR 3-13437941121>

Applies to:

Symptoms

OAM.SYSTEM MeasArrayed reports often show in the System.CPU_CoreUtilPct_Peak value hitting 100% for one or more Index entries (CPU Core number) on Operation, Administration, and Maintenance (OAM) servers, often appearing at regular intervals.

OAM.SYSTEM MeasSimple reports will show System.CPU_UtilPct_Peak values high, often above 80% at regular intervals.

No EventID 31215 'Process Resources Exceeded' or other utilization related alarms in the Alarm and Event reports are typically observed.

On the OAM server via command line, 'top -M -n 5' (will produce 5 samples over about 10s) command output shows Imysqld as a leading CPU consumer, often above 150%.

Changes

Possible changes include:  Feature activation; Software upgrade; Implementation of a kpi or report monitoring strategy.  Often the condition is noticed well after the change was made, since it mainly manifests in OAM.SYSTEM measurement reporting.

Cause

Typical OAM architecture is a virtual server with four CPU cores assigned.  Both types of OAM - Site and Network - act as collectors and reporting engines for the servers within their domain.  One of their principal functions is Measurement and KPI reporting.  Data is continuously collected from the subordinate servers for reporting, however these reports are only generated via (1) manual request through the OAM Graphical User Interface (GUI), or (2) automatically at scheduled intervals set up by the user through the OAM GUI's Export capabilities. 

Report generation is a CPU intensive task.  Therefore the more reports chosen by the user to generate, and the larger the server count reporting to that OAM, the higher the burden on the OAM CPU.  Additional conditions that can contribute to the high CPU utilization are the frequency the reports are generated, the granularity of each report's time interval (5-min is the most granular thus highest processing burden), and the number of configured Diameter connections (for certain connection-related reports).

Generally, sites where this condition has been raised as a concern have been found to have one or more of the following factors:

1. ALL possible reports have been selected to be generated.
2. Report generation is scheduled frequently, often all starting at the same time.
3. Report granularity is high (frequent interval)
4. Site deployment is large in terms of server count, feature use, and Diameter connection count.
5. Several scheduled task types (e.g. KPI, Alarm, Measurement, etc.) are initiated within the same time window.

Solution

The elevated CPU utilization is caused by scheduled activity (i.e. Measurement, KPI, etc. report generation and export), and the level determined mostly by the number and granularity of reports being generated.   If the level of CPU utilization is a concern, the following actions are recommended:

1. Review the use of Measurement and KPI data reported/exported from the DSR, identify those unused or unnecessary and remove them from the scheduled Export task. Refer to the DSR Operations, Administration, and Maintenance User's Guide on how to edit a scheduled task.
2. If a large number of Measurement reports are required, create two or three separate groups for reporting and schedule them at different times.
3. Consider the required granularity, and adjust individual reports to fewer intervals if possible.
4. Review all scheduled tasks and make adjustments to have them initiate at different start times.

This solution has no attachment