Diameter Signaling Router (DSR): Starting Integrated Diameter Intelligence Hub Mediation (IDIH-MED) Fails After 8.1 Upgrade; boot.log indicates "prod.start ABORTING: bad exit code..."

Asset ID:	1-72-2378675.1
Update Date:	2018-03-26
Keywords:

Solution Type Problem Resolution Sure

Solution 2378675.1 : Diameter Signaling Router (DSR): Starting Integrated Diameter Intelligence Hub Mediation (IDIH-MED) Fails After 8.1 Upgrade; boot.log indicates "prod.start ABORTING: bad exit code..."

Applies to:

Oracle Communications Diameter Signaling Router (DSR) - Version DSR 8.0.0 to DSR 8.2.0 [Release DSR 8.0]
Tekelec

Symptoms

Starting/restarting Mediation VM failed reporting IDB database errors. May also see Major 19800 and Minor 11511 ComAgent alarms also appear on installed DA-MPs.

The /var/log/boot.log shows the following error (date/time generalized in example):

<snip prior entries...>
idbinit: set replication checkpoint (after graceful shutdown/startup)

MM/DD/YYYY HH:MM:SS idbinit#31000{}
** GN_BADOP/FTL AppEventLog seems misconfigured (expecting ML w 2+ subparts) [MergeSubpartTool.cxx:1005]
^^ MergeSubpartTool::getMergeTbls(type=ML) [MergeSubpartTool.cxx:1015]
^^ MergeSubpartSingleton::makeHarmony(doWait=true) [MergeSubpartTool.cxx:236]
^^ MergeSubpartSingleton::repairMapUsingNodeBlk() [MergeSubpartTool.cxx:733]
^^ MergeSubpartTool::auditUsingAppEventLog() [MergeSubpartTool.cxx:1080]
^^ [24014:idbinit.cxx:716]

************** !!!!!!!!!!!!!!!!!!! *******************
***
*** prod.start ABORTING: bad exit code (256): idbinit -fwtkL1
*** NOTE: manual recovery may be required
***
************** !!!!!!!!!!!!!!!!!!! *******************

Changes

Problem appeared after upgrade to IDIH 8.1 with the server restart step, but could also be encountered later following virtual machine (VM) server reboot or restart.

Cause

The system determined AppEventLog table is not structured as expected by the software.
The true cause per related Bug is TablDef.repPlanId field is set incorrectly for table AppEventLog

Solution

Apply workaround below; upgrade to DSR 8.2 or later release for permanent solution.

1. Login to mediation server as "admusr" and change the user to "tekelec"

[admusr@idih-med ~]$ sudo su - tekelec

2. Change table parameter using the following command:

NOTE: 'tekelec' user prompt will look something like "idih-med:/usr/TKLC/xIH" with no brackets.
Below brackets [] are just for delineation purposes in this article

[idih-med:/usr/TKLC/xIH] iset -frepPlanId='ML' TableDef where "name='AppEventLog'"

Output should indicate 1 records changed. Example portion of output to look for:
=== changed 1 records ===

3. Restart the mediation process:

[idih-med:/usr/TKLC/xIH] prod.stop;prod.start

The IDIH Mediation server should start correctly and related comagent alarms clear.

References

<BUG:27137104> - MAJOR ALARMS UNABLE TO CONNECT VIA COMAGENT TO REMOTE DIH SERVER WITH HOSTNAME

Attachments

This solution has no attachment