Oracle Database Appliance
 
Oracle Database Appliance Software
 

PLA-Support>Eng Systems>Exadata/ODA/SSC>Oracle Database Appliance>DB: ODA_EST
 

Applies to:

Symptoms

NTP is not synchronizing correctly on a version 2.2 ODA, which is causing RAC to be off by about 5 minutes at the end of the day.

Cause

You may not be using the newest ODA software / firmware.
If your ODA is using OAK version 2.2.0.0.0 and your ILOM version is older than 3.0.14.13.a r70764 (for example 3.0.14.13 r66865), you may hit this issue.

 
The Linux kernel has different implementations of lost tick compensation depending on whether the timer device is the TSC or the PM timer, and whether the kernel is i386 (32bit) or x86_64 (64bit). The implementations using the TSC are much more prone to overcompensation.  This TSC deficiency is corrected in newer ILOM firmware versions.  This also allows the workaround (below) to force the use of the PM timer.

You can see what implementation of lost tick compensation that you are currently using with:

     # cat /sys/devices/system/clocksource/clocksource0/current_clocksource
        tsc

Solution

1. Temporary workaround is the "clocksource=acpi_pm" kernel argument, followed by an OS reboot to implement:

      kernel /vmlinuz-* ... notsc clocksource=acpi_pm

2. Permanent solution is newer firmware, included with new download of 2.2 ODA OS image (or newer). There are three ways to achieve this:

      a.) Infrastructure patching, using the ODA 2.2.0.0 patchset (which includes ILOM 3.0.14.13.a r70764):

           Patch 13858874, following the README instructions from section "For all other existing deployments"

                 # ./oakcli update -patch 2.2.0.0.0 --infra

      b.) ODA OS re-imaging, using NOTE 1373599.1, “Oracle Database Appliance Bare Metal Restore Procedure”

      c.) Manual firmware update. Use Keyboard Controller Style (KCS) interface to apply ILOM/BIOS patches as shown below:

[root@odabox1 ~] cd /opt/oracle/oak/pkgrepos/thirdpartypkgs/Firmware/Ilom/SUN/541-4116-03/3.0.14.13.a_r70764/Base
[root@odabox1 Base]# ls
ILOM-3_0_14_13_a_r70764-Sun_Fire_X4370M2.pkg
[root@odabox1 Base]# ipmiflash -v write ILOM-3_0_14_13_a_r70764-Sun_Fire_X4370M2.pkg force script config delaybios warning=0

-- Wait 3-4 minutes to allow ILOM/SP to finish rebooting after patching and issue the following command to reboot the entire ODA system:

[root@odabox1 Base]# ipmiflash chassis power cycle

References

This solution has no attachment