Exadata server Reboots Unexpectedly;

Asset ID:	1-72-2292617.1
Update Date:	2017-08-08
Keywords:

Solution Type Problem Resolution Sure

Solution 2292617.1 : Exadata server Reboots Unexpectedly;

Applies to:

Exadata Database Machine V2 - Version All Versions and later
Information in this document applies to any platform.

Symptoms

Exadata node reboots unexpectedly, producing a vmcore which shows a backtrace like the following:

crash> bt
PID: 0 TASK: ffff883f054c6580 CPU: 23 COMMAND: "kworker/0:1"
#0 [ffff88407f4e3900] machine_kexec at ffffffff8103b40d
#1 [ffff88407f4e3970] crash_kexec at ffffffff810b98c5
#2 [ffff88407f4e3a40] oops_end at ffffffff815077d8
#3 [ffff88407f4e3a70] die at ffffffff8101868b
#4 [ffff88407f4e3aa0] do_trap at ffffffff815071b0
#5 [ffff88407f4e3af0] do_invalid_op at ffffffff810165e5
#6 [ffff88407f4e3b90] invalid_op at ffffffff8150fcfb
[exception RIP: rds_ib_dev_put+50]
RIP: ffffffffa0410662 RSP: ffff88407f4e3c40 RFLAGS: 00010286
RAX: 0000000080000009 RBX: ffff883f031ed580 RCX: 0000000000000001
RDX: 00000000000004cc RSI: 0000000000000000 RDI: ffff883f031edc80
RBP: ffff88407f4e3c40 R8: 0000000000000000 R9: 00000000ffffffff
R10: ffff883142bf74c0 R11: 00000000000000bc R12: 0000000000000000
R13: ffff883f031edc80 R14: 0000000000000292 R15: 0000000000000000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
#7 [ffff88407f4e3c48] rds_ib_free_mr at ffffffffa041b3d8 [rds_rdma]
#8 [ffff88407f4e3c78] rds_destroy_mr at ffffffffa03f35d2 [rds]
#9 [ffff88407f4e3ca8] rds_rdma_unuse at ffffffffa03f38ba [rds]
#10 [ffff88407f4e3ce8] rds_recv_incoming_exthdrs at ffffffffa03ef0d3 [rds]
#11 [ffff88407f4e3d48] rds_recv_incoming at ffffffffa03ef25f [rds]
#12 [ffff88407f4e3d78] rds_ib_process_recv at ffffffffa0415ff3 [rds_rdma]
#13 [ffff88407f4e3dc8] rds_ib_recv_cqe_handler at ffffffffa04178d2 [rds_rdma]
#14 [ffff88407f4e3e08] poll_cq at ffffffffa0412fc6 [rds_rdma]
#15 [ffff88407f4e3e58] rds_ib_tasklet_fn_recv at ffffffffa041395e [rds_rdma]
#16 [ffff88407f4e3eb8] tasklet_action at ffffffff810767aa
#17 [ffff88407f4e3ed8] __do_softirq at ffffffff81075c59
#18 [ffff88407f4e3f38] call_softirq at ffffffff8150ff7c
#19 [ffff88407f4e3f50] do_softirq at ffffffff810172e5
#20 [ffff88407f4e3f70] irq_exit at ffffffff8107658b
#21 [ffff88407f4e3f80] do_IRQ at ffffffff815107e9
--- ---
#22 [ffff883f054c9e30] ret_from_intr at ffffffff81506a93
RIP: ffffffffffffff44 RSP: 0000000000000246 RFLAGS: 00000010
RAX: 0000000000000000 RBX: ffff883f054c9ef8 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff883f054c9fd8
RBP: ffffffff81506a8e R8: ffff88407f4ee3c0 R9: ffff88407f4ee410
R10: 0000000000000000 R11: ffff88407f4ee500 R12: 0000000000000000
R13: 00000001810982c6 R14: 000f60030ffbfa00 R15: 0000000000000000
ORIG_RAX: 0000000000000000 CS: ffffffff8101de79 SS: ffff883f054c9ee8
bt: WARNING: possibly bogus exception frame

Cause

This issue is due to a known kernel defect - BUG 21288594 - RDS: PANIC IN RDS_IB_DEV_PUT

Solution

Applying the following ARU should resolve the issue:
ARU 21134761

References

<BUG:21288594> - RDS: PANIC IN RDS_IB_DEV_PUT

Attachments

This solution has no attachment