Oracle Exadata Storage Server Software
 
SPARC SuperCluster T4-4 Half Rack
 
SPARC SuperCluster T4-4
 
Oracle SuperCluster M6-32 Hardware
 
Oracle SuperCluster T5-8 Half Rack
 
Oracle SuperCluster T5-8 Full Rack
 
SPARC SuperCluster T4-4 Full Rack
 

PLA-Support>Eng Systems>Exadata/ODA/SSC>SPARC SuperCluster>DB: SuperCluster_EST
 
_Old GCS Categories>ST>Server>Engineered Systems>SPARC SuperCluster>Install
 

This document provides reference and steps to handle the Leap Second event on Oracle SuperCluster configurations, conveying descriptions on available options and mechanisms, and enabling the customer to properly setup and verify NTP services/clients.

In this Document

Goal

Solution

Background

Prerequisites and Checks

How to check NTP services and software version

On Database LDOMs/Zones:

How to check RAC health (LDOMs)

How to check RAC health (Zones)

Exadata Storage Cells interaction-Leap Second on Linux

Leap Second NTP setup steps:

NTPv4 - Using a leap second file (Solaris 11):

What to look for on Jun/30 or Dec/31

NTPv3 - Setting leap bits via ntpq (Solaris 10):

External References

References

Applies to:

Goal

Provides references, background and steps to configure NTP to handle Leap Second events on Oracle SuperCluster.

Solution

Background

The U.S. National Institute of Standards and Technology (NIST) is publishing a file which contains a table of past and upcoming leap seconds. This file can be used by ntpd to become aware of leap second announcements and thus be able to handle leap seconds gracefully or as a guide to set older NTP versions to apply the required correction when scheduled.

Graceful handling of leap seconds implies applying the leap second locally at the appropriate time, rather than having the clock off by one second until the discrepancy with sources is noticed and corrected at a later time. ntpd will gracefully handle leap seconds which it knows about in advance

Reference: http://support.ntp.org/bin/view/Support/ConfiguringNTP#Section_6.14

Prerequisites and Checks

How to check NTP services and software version

This sentence will provide the status of the NTP service

# svcs ntp
STATE          STIME    FMRI
online          7:26:44 svc:/network/ntp:default

Software version follows the same sequence as NTP definitions and standards, basically the code is a snapshot taken from the NTP.org site.

  Finally a verification on peer NTP servers configured

On Database LDOMs/Zones:

SuperCluster uses non-global zones for RAC and all share the global zone kernel, therefore there is only one date/time setup, controlled by the global zone only. The NTP service can't be enabled in non-global zones, the reason being that by default the change date/time privilege is unavailable: all of them will refer to the global zone (LDOM) time/date and its NTP configuration as an extension.

That leaves Real Application Clusters working on Cluster Time Synchronization Services (CTSS) observer mode, when configured in Zones; which is safe since the lower layer of synchronization is essentially NTP.

How to check RAC health (LDOMs)

Run cluvfy on each RAC cluster, using the oracle or grid-owner user. The -noctss parameter is useful since we're interested on verifying NTP only.

How to check RAC health (Zones)

Run cluvfy on each RAC cluster/set of zones, remember use the grid-owner OS user or oracle, depending on your setup. This will test both for NTP and CTSS.

Clusterware tolerates a 15minute gap between members clocks, according to note 786048.1

Exadata Storage Cells interaction-Leap Second on Linux

During our tests no impact was perceived on communication between Database Nodes and Storage Cells, despite the fact that Linux Leap Second implementation differs from the one provided by Solaris, the loose level of coupling (time synchronization perspective) between Database Nodes and Storage Cells allows for continued stability.

Leap Second NTP setup steps:

While the NTP protocol addresses the leap second event as a cascading event, that propagates from the top of the hierarchy down to clients/servers and reaching the end on sole clients. It's recommended to take provisions within your infrastructure to address the event on your NTP servers and those servers that consume the service, with local NTP configuration for that purpose.

NOTE: this steps must be performed on all NTP servers providing service, as well as all Solaris LDOMs

NTPv4 - Using a leap second file (Solaris 11):

Get the file from either ftp://time.nist.gov/pub/ or ftp://tycho.usno.navy.mil/pub/ntp/ and upload/copy it to /etc/inet

Go to directory /etc/inet and edit ntp.conf and include (or uncomment) this line leapfile /etc/inet/ntp.leap

Then create a symbolic link, restart the NTP service and verify is up and running.

 Immediatelly after the service is up, verify the file was properly "installed" querying the service using ntpq.

 or look for leap_armed within the status string

What to look for on Jun/30 or Dec/31

24 hours before the event, the NTP service will internally activate the "awareness" mechanisms, providing a couple of useful indicators in order to verify readiness. The status string will change from leap_none to leap_add_sec and the leap variable will change from 00 to 01.

NTPv3 - Setting leap bits via ntpq (Solaris 10):

The NTP software provided by default is version 3 and there is no support for leap file. Leap second is handled using leap bits instead.

You can set leapwarning bit up to 31 days before the Leap Second event, that would be May/31-Jun/30 or Dec/1-Dec/31 ranges; while the leapindicator bit can only be set 24 hours before the event, that is on Jun/30 or Dec/31.

 Note: More information on Leap Second bits can be found on MOS Document 1474401.1.

NTP versions shipped so far with Solaris 11 SuperCluster software bundle and QFSDPs started with 4.2.5, up to 4.2.8 the latest.

External References

NOTE: This are external references, highly recommended reading.

http://www.eecis.udel.edu/~mills/leap.html

References

This solution has no attachment