ODAVM: Upgrading to12.1.2.11/12 After the oda_base Startup the GI Startup Fails on Both Nodes Due to IB Failure

Asset ID:	1-72-2300740.1
Update Date:	2018-02-20
Keywords:

Solution Type Problem Resolution Sure

Solution 2300740.1 : ODAVM: Upgrading to12.1.2.11/12 After the oda_base Startup the GI Startup Fails on Both Nodes Due to IB Failure

Applies to:

Oracle Database Appliance X5-2 - Version All Versions to All Versions [Release All Releases]
Oracle VM - Version 3.2.6 to 3.2.6 [Release OVM32]
Information in this document applies to any platform.
The issue may or may not get solve by restart dom0.

Symptoms

Upgrading the X6-2 HA and X5-2 ODA Virtualized Platform Server to 12.1.2.11/12.1.2.12
After the oda_base reboot the GI fails to come up on both nodes.
The IB interconnection IP address on oda_base 192.168.16.27/28 is not pingable.
Unable to bring up the IB interface at oda_base
More symptoms include:

# ibstat
does not return anything

# service rdma status
Configured IPoIB interfaces: ib0 ib1 ibbond0

Currently active IPoIB interfaces: none <<<<<<<<<<

# modprobe ib_ipoib
WARNING: All config files need .conf: /etc/modprobe.d/modprobe.conf.backup_stig, it will be ignored in a future release.

# ifup ib0
mlx4_core device ib0 does not seem to be present, delaying initialization.

# ifup ib1
mlx4_core device ib1 does not seem to be present, delaying initialization.

Changes

Upgrade the ODA 5-2 or X6-2HA VM (Virtualized Platform) to 12.1.2.11 (UEK4)/12.1.2.12(UEK4)

Cause

The oda_base is using UEK4 which IB passthrough will need much more IRQ which cause the IB interface cannot startup.

Solution

1) view

grep -i dom0_mem= /boot/grub/grub.conf

     # all kernel /xen.gz dom0_mem=4096M crashkernel=256M@64M extra_guest_irqs=64,2048 nr_irqs=2048
.       ...
        ...

2) Edit

Change the value

extra_guest_irqs=64,2048

to extra_guest_irqs=256,2048

3) Then reboot the whole machine (dom0).

X5-2 ODA_BASE Dom0 IB Upgrade patchbundle 12.1.2.11 ODAVP patch UEK4 extra_guest_irqs OVM

References

<NOTE:2207843.1> - Oracle VM Server Installation on Sun Fire X4800 or Fujitsu RX2540 M4
<NOTE:1953714.1> - Oracle VM 3.2.8 installation hangs on a Sun Server X4-4
<NOTE:2154480.1> - ODA VM : How to Resize the Memory Allocated by Dom#0 in ODA VM Configurations

Attachments

This solution has no attachment