OCEEMS Error When Trying to Run Any Command "-bash: fork: retry: Resource temporarily unavailable"

Asset ID:	1-72-2338449.1
Update Date:	2018-01-16
Keywords:

Solution Type Problem Resolution Sure

Solution 2338449.1 : OCEEMS Error When Trying to Run Any Command "-bash: fork: retry: Resource temporarily unavailable"

Applies to:

Oracle Communications EAGLE (Hardware) - Version E5-MS 45.0 and later
Tekelec

Symptoms

Following error is observed while trying to su to OCEEMS administrative user emsadm:

[root@oceems1 ~]# su - emsadm
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: retry: Resource temporarily unavailable
-bash: fork: Resource temporarily unavailable

Cause

limit for the no. of open files (ulimit -n) and maximum user processes (ulimit -u) is too low
This can be checked by command ulimit -a :

[emsadm@oceems1 bin]$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 128175
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 10240
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 1024
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[emsadm@oceems1 bin]$

Refer to customer documentation >> Install/Upgrade Guide >> Section 1.4.2 Disk Space and Open File Limit
It requires the max open files to be set to 65536
https://docs.oracle.com/cd/E88599_01/doc.465/E88601_rev_1.pdf

Solution

Copy following file before making changes to it:

cp /etc/security/limits.conf /etc/security/limits.orig.conf
Edit the file /etc/security/limits.conf

vi /etc/security/limits.conf
Add following lines to the file for user emsadm:

  emsadm soft nofile 65536
  emsadm hard nofile 65536
  emsadm soft nproc 10240
  emsadm hard nproc 10240
If active/stanby setup is being used, perform actions on both servers.
Restart the E5MS application. (If active/standby, first restart application on standby and then restart on active so that roles remain same after restart.)

service e5msService stop
service e5msService start

References

<NOTE:2252859.1> - Oracle Linux: running various commands returns: -bash: Fork: Resource Temporarily Unavailable

Attachments

This solution has no attachment