Oracle Communications EAGLE (Software)
 

PLA-Support>Sun Systems>CommsGBU>Global Signaling Solutions>SN-SND: Tekelec EMS
 

In this Document

Symptoms

Changes

Cause

Solution

Created from <SR 3-9620334291>

Applies to:

Symptoms

Issue Clarification

User reported that the Element Management System (EMS) Fault Management is not opening.
Checked the EMS application and found that the EMS application was not running

SunOS[netboss]/u01/app/netboss/sga/debugfiles% $ANMDIR/admin/bin/netboss_init check
0 instance(s) of roc were found. 1 instance(s) expected.
0 instance(s) of alertman were found. 1 instance(s) expected.
0 instance(s) of gatewaymgr were found. 1 instance(s) expected.
0 instance(s) of imd were found. 1 instance(s) expected.
0 instance(s) of loghistd were found. 1 instance(s) expected.
master_script is running.
0 instance(s) of sgasecurity were found. 1 instance(s) expected.
0 instance(s) of sgaalert were found. 1 instance(s) expected.
0 instance(s) of proengine were found. 1 instance(s) expected.
0 instance(s) of check-failover were found. 1 instance(s) expected.
0 instance(s) of nbtf were found. 1 instance(s) expected.
SunOS[netboss]/u01/app/netboss/sga/debugfiles%



The loss of functionality
EMS Cannot be used for Eagle Alarm Monitoring

cusotmer impact
Customer cannot use EMS for Eagle Alarm Monitoring


Include specific error messages (error number with syntax)
roc, alertman, imd, loghistd, gatewaymgr are important processes for EMS application but none was running. (as show above)
When trying to stop/start the oracle DB; following error was found:
Starting /u01/app/oracle/products/9.2/bin/tnslsnr: please wait...

TNSLSNR for Solaris: Version 9.2.0.8.0 - Production
System parameter file is /u01/app/oracle/products/9.2/network/admin/listener.ora
Log messages written to /u01/app/oracle/products/9.2/network/log/listener.log
Error listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=)(PORT=1521)))
TNS-12545: Connect failed because target host or object does not exist
TNS-12560: TNS:protocol adapter error
TNS-00515: Connect failed because target host or object does not exist


Listener failed to start. See the error message(s) above...

Following error was found in Oracle log:
SunOS[netboss]/u01/app/oracle/products/9.2/admin/NBPRD/bdump% tail -100 alert_NBPRD.log
ORA-1631: max # extents 4096 reached in table NETBOSS.ALERT_TABLE
ORA-1631: max # extents 4096 reached in table NETBOSS.ALERT_TABLE
ORA-1631: max # extents 4096 reached in table NETBOSS.ALERT_TABLE
ORA-1631: max # extents 4096 reached in table NETBOSS.ALERT_TABLE
ORA-1631: max # extents 4096 reached in table NETBOSS.ALERT_TABLE
 

What is the expected or acceptable result?
All EMS processes should be running. Oracle should be running with lisntener status as started.

Changes

 What about the system has recently changed?
According to User, no recent changes to system were made

Cause

The Oracle DB listener could not be started because the system hostname has been changed.
SunOS% hostname
SunOS

/etc/hosts file shows that system hostname should be EMS-UMR

SunOS% cd /etc/
SunOS% cat hosts
#
# Internet host table
#
127.0.0.1 localhost
::1 localhost
172.##.32.106 EMS-UMR nbems1 loghost
172.##.32.107 EMS-UMR1
10.15.##.104 pugsley
The issue is due to:
1. Increase maximum extents for NETBOSS.ALERT_TABLE from 4096 to 8192
2. wrong hostname set in system.

Solution

STEP 1. Increase max extents for the table NETBOSS.ALERT_TABLE

SunOS[netboss]/u01/app/oracle/products/9.2/admin/NBPRD/bdump% sqlplus netboss/netboss

SQL*Plus: Release 9.2.0.8.0 - Production on Wed Sep 17 21:36:31 2014

Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.


Connected to:
Oracle9i Release 9.2.0.8.0 - Production
JServer Release 9.2.0.8.0 - Production

NETBOSS@NBPRD:SunOS> alter table ALERT_TABLE storage (maxextents 8192)
  2 ;

Table altered.

NETBOSS@NBPRD:SunOS> exit



STEP 2. Change the hostname to correct value
As user root:
# hostname EMS-UMR
# hostname
EMS-UMR

STEP 3: Start Oracle
As user root:
# /etc/init.d/oracle start

SQL*Plus: Release 9.2.0.8.0 - Production on Thu Sep 18 00:53:19 2014

Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.

nolog> Connected to an idle instance.
nolog> ORACLE instance started.

Total System Global Area 296309216 bytes
Fixed Size 455136 bytes
Variable Size 251658240 bytes
Database Buffers 33554432 bytes
Redo Buffers 10641408 bytes
Database mounted.
Database opened.
nolog> Disconnected from Oracle9i Release 9.2.0.8.0 - Production
JServer Release 9.2.0.8.0 - Production

Database "NBPRD" warm started.

LSNRCTL for Solaris: Version 9.2.0.8.0 - Production on 18-SEP-2014 00:53:25

Copyright (c) 1991, 2006, Oracle Corporation. All rights reserved.

Starting /u01/app/oracle/products/9.2/bin/tnslsnr: please wait...

TNSLSNR for Solaris: Version 9.2.0.8.0 - Production
System parameter file is /u01/app/oracle/products/9.2/network/admin/listener.ora
Log messages written to /u01/app/oracle/products/9.2/network/log/listener.log
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=EMS-UMR)(PORT=1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias LISTENER
Version TNSLSNR for Solaris: Version 9.2.0.8.0 - Production
Start Date 18-SEP-2014 00:53:25
Uptime 0 days 0 hr. 0 min. 0 sec
Trace Level off
Security OFF
SNMP OFF
Listener Parameter File /u01/app/oracle/products/9.2/network/admin/listener.ora
Listener Log File /u01/app/oracle/products/9.2/network/log/listener.log
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=EMS-UMR)(PORT=1521)))
Services Summary...
Service "NETBOSS" has 1 instance(s).
  Instance "NBPRD", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully


STEP 4: Start EMS
As user root:
# /etc/init.d/netboss start
0 instance(s) of roc were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/roc >> /u01/app/netboss/log/roc.dlog 2>> /u01/app/netboss/log/roc.derror_log &
0 instance(s) of alertman were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/alertman >> /u01/app/netboss/log/alertman.dlog 2>> /u01/app/netboss/log/alertman.derror_log &
0 instance(s) of gatewaymgr were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/gatewaymgr start >> /u01/app/netboss/log/gateway.dlog 2>> /u01/app/netboss/log/gateway.derror_log &
0 instance(s) of imd were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/imd >> /u01/app/netboss/log/imd.dlog 2>> /u01/app/netboss/log/imd.derror_log &
0 instance(s) of loghistd were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/loghistd >> /u01/app/netboss/log/loghistd.dlog 2>> /u01/app/netboss/log/loghistd.derror_log &
0 instance(s) of master_script were found. 1 instance(s) expected.
Executing: /u01/app/netboss/local/lib/check_scripts/master_script master >> /u01/app/netboss/log/master_script.dlog 2>> /u01/app/netboss/log/master_script.derror_log &
0 instance(s) of sgasecurity were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/sgasecurity >> /u01/app/netboss/log/sgasecurity.dlog 2>> /u01/app/netboss/log/sgasecurity.derror_log &
0 instance(s) of sgaalert were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/sgaalert >> /u01/app/netboss/log/sgaalert.dlog 2>> /u01/app/netboss/log/sgaalert.derror_log &
0 instance(s) of proengine were found. 1 instance(s) expected.
Executing: /u01/app/netboss/bin/proengine >> /u01/app/netboss/log/proengine.dlog 2>> /u01/app/netboss/log/proengine.derror_log &
0 instance(s) of check-failover were found. 1 instance(s) expected.
Executing: /u01/app/netboss/admin/bin/check-failover >> /u01/app/netboss/log/failover.dlog 2>> /u01/app/netboss/log/failover.derror_log &
0 instance(s) of nbtf were found. 1 instance(s) expected.
Executing: /u01/app/netboss/local/bin/nbtf >> /u01/app/netboss/log/trapforwarder.dlog 2>> /u01/app/netboss/log/trapforwarder.derror_log &
#

STEP 5:
Check EMS Fault Management is working and alarms are appearing

This solution has no attachment