PIC Crash of IxpBuild Process Because of Memory Limit of 32 Bits Operating System (E

Asset ID:	1-72-1991330.1
Update Date:	2016-03-26
Keywords:

Solution Type Problem Resolution Sure

Solution 1991330.1 : PIC Crash of IxpBuild Process Because of Memory Limit of 32 Bits Operating System (E_NOMEM: cannot shmat)

Applies to:

Oracle Communications Performance Intelligence Center (PIC) Software - Version 7.1 to 9.0.4 [Release 7.0 to 9.0]
BNS Platform Hardware
Information in this document applies to any platform.

Symptoms

xDR Session stopped or there is a gap in xDR generated by IxpBuild.

Cause

IxpBuild process is continously logging this (ERROR: E_NOMEM: cannot shmat):

0320:064925.264 DTS2 DtsProducerBuffer::Flush addRec failed : ERROR: E_NOMEM: cannot shmat [CoreD.cxx:365]
^^ shmat(shmid=1661763928, shmaddr=0, shmflg=0) [CoreD.cxx:367]
^^ IprFragShMem::createShSeg(size=2097152, readOnly=0, useHugePages=0) [IprFragShMem.cxx:2691]
^^ IprFragShMem::createFragShMem(isBig=0, useHugePages=0, fragSizeK=2048, tblCnt=7, subtblCnt=4) [IprFragShMem.cxx:468]
^^ IdbDict::mkEmptyShFrag(partName=DtsBlockPart, subpart=91, fragInst=0xffffffff) [mkEmpFrag.cxx:76]
^^ IdbPart::initEmptyFrag(part=DtsBlockPart, shmKey=-1) [IdbPart.cxx:1669]
^^ IdbPart::getNewFrag(begTime=1428785249280, endTime=1428785249280) [IdbPart.cxx:1424]
^^ IdbFragIter::switchFrag(newFrag=0, doCopyAutoIncr=1) [IdbFragIter.cxx:614]
^^ current frag=0x39b3 20140320.061702.0000.DtsBlockPart.0091,28 [IdbFragIter.cxx:618]
^^ IdbIter::addRecSelect(void *xRec) [IdbIter.cxx:2467]
, Stream : 2680155 [30932/DtsProducerBuffer.cpp:539]

Solution

Because of 32 bits operating system limits, IxpBuild cannot allocate more memory.

Short term workaround

The IXP application needs to be restarted.

As cfguser run the following:

prod.dbdown
prod.start

Long term solution

Two options exist:

Upgrade to 64 bits OS (PIC 10.1 or newer)
Split traffic in two or more IxpBuild to reduce load per process

Attachments

This solution has no attachment