Diameter Intelligence Hub (DIH) ProTrace Sessions Stopped Updating and Exporting

Asset ID:	1-72-2062525.1
Update Date:	2018-04-27
Keywords:

Solution Type Problem Resolution Sure

Solution 2062525.1 : Diameter Intelligence Hub (DIH) ProTrace Sessions Stopped Updating and Exporting

Applies to:

Oracle Communications Diameter Signaling Router (DSR) - Version DSR 5.0 and later
Information in this document applies to any platform.

Symptoms

DIH sessions stop:

No new xDR are created
No new xDR are exported.

Changes

Cause

Build process has allocated maximum allowed memory.

This can be confirmed by looking at IxpBuild log(tr.tail IxpBuildxxxxx):

1001:103906.431 DTS2 DtsProducerBuffer::Flush addRec failed : ERROR: E_NOMEM: cannot shmat [CoreD.cxx:365]
^^ shmat(shmid=60031292, shmaddr=0, shmflg=0) [CoreD.cxx:367]
^^ IprFragShMem::createShSeg(size=2072576, readOnly=0, useHugePages=0) [IprFragShMem.cxx:2869]
^^ IprFragShMem::createFragShMem(isBig=0, useHugePages=0, fragSizeK=2024, tblCnt=7, subtblCnt=4) [IprFragShMem.cxx:616]
^^ IdbDict::mkEmptyShFrag(partName=DtsBlockPart, subpart=90, fragInst=0xffffffff) [mkEmpFrag.cxx:76]
^^ IdbPart::initEmptyFrag(part=DtsBlockPart, shmKey=-1) [IdbPart.cxx:1701]
^^ IdbPart::getNewFrag(begTime=1478344638788, endTime=1478344638788) [IdbPart.cxx:1455]
^^ IdbFragIter::switchFrag(newFrag=0, doCopyAutoIncr=1) [IdbFragIter.cxx:634]
^^ current frag=0x147 20151001.102019.1880.DtsBlockPart.0090,25 [IdbFragIter.cxx:638]
^^ IdbIter::addRecSelect(void *xRec) [IdbIter.cxx:2467]
, Stream : 16218 [1861/DtsProducerBuffer.cpp:538]

Solution

Verifications

Maximum Total Shared Memory

Check amount of Maximum Total Shared Memory:

$ shl.show
Maximum No. of Shared Memory Segments = unlimited
Maximum Total Shared Memory Size = 1500M

Current Shared Memory Segment Count = 330
Current Shared Memory Total Size = 869M
Recent Shared Memory Max Size = 1125M
Kernel Maximum Segment Size = 2097152K
SeqNum = 1123664

Procedure must be applied only on:

32 bits OS
server having 8GB or more
when amount of Maximum Total Shared Memory is 1500 as shown above.

Software version

Procedure can be applied on any IXP 8.x version, but it is highly recommended to upgrade IXP to 8.0.0-37.1.0. This is similar to OCPIC bug 19110014 which is fixed by a COMCOL fix.

On IXP server, "rpm -qi TKLCixp" to verify.

Procedure

Run following procedure:

Connect on IXP as cfguser
Increase Maximum Total Shared Memory Size:
shl.set -m 3000
Stop product:
prod.dbdown
Start up product:
prod.start

This is reset by prod.clobber, thus it must be reapplied after.

Document based on technical bulletin TN003460.

References

<BUG:19110014> - [232076]UPGRADE IS NOT COMPLET EVEN IF SERVER REBOOTED AUTOMATICALY

Attachments

This solution has no attachment