kernel: qemu-kvm invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 resulting in unexpected guest shut-off

Asset ID:	1-72-2286383.1
Update Date:	2017-07-13
Keywords:

Solution Type Problem Resolution Sure

Solution 2286383.1 : kernel: qemu-kvm invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 resulting in unexpected guest shut-off

Applies to:

Oracle Communications Diameter Signaling Router (DSR) - Version DSR 5.0 and later
Information in this document applies to any platform.

Symptoms

Symptoms can be the expected shutdown of a virtual guest on TVOE host or in general another process on a server crashing

Cause

Looking at the logs /var/log/messages we see the following

kernel: qemu-kvm invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0

This is the same signature and behavior as the bug described below:
https://bugzilla.redhat.com/show_bug.cgi?id=891653

Additional searches showed the following noteworthy results

https://mosemp.us.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrl-state=r952joswu_76&_afrLoop=408395059055506#SYMPTOM

Out_Of_Memory Happens With Incorrect tmpfs Setting (Doc ID 2165914.1)

terminating the instance due to error 471 Out-Of-Memory(OOM) Killer Crashes Oracle Database (Doc ID 1622379.1)

From <https://mosemp.us.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrl-state=r952joswu_76&_afrLoop=408395059055506#CAUSE>

The above shows cases in other products where the server is running out of memory for various reasons and a critical process is terminated by oom-killer.

In this particular case the process that is terminated is "qemu-kvm" which causes the virtual guests on a TVOE to be shut-off.

Some additional info on OOM-killer:

Linux: Out-of-Memory (OOM) Killer (Doc ID 452000.1)

From <https://mosemp.us.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=409201887343186&parent=DOCUMENT&sourceId=1622379.1&id=452000.1&_afrWindowMode=0&_adf.ctrl-state=r952joswu_417>

Solution

Consider adding additional RAM to the server or migrating to a server with higher memory capacity.

Attachments

This solution has no attachment