SPARC T3-1
 
SPARC T3-4
 
Sun SPARC Enterprise T5220 Server
 
SPARC T4-2
 
Sun SPARC Enterprise T5240 Server
 
Netra SPARC T4-2 Server
 
SPARC T5-8
 
Netra SPARC T4-1 Server
 
Sun Netra T5220 Server
 
SPARC T5-4
 
SPARC T4-1
 
SPARC T5-2
 
Sun Netra T5440 Server
 
SPARC T3-2
 
Netra T3-1
 
Sun SPARC Enterprise T5440 Server
 
SPARC T4-4
 

PLA-Support>Sun Systems>SPARC>CMT>SN-SPARC: T5
 

Applies to:

Purpose

This doc indicates the items which control the time that Solaris initially boots with on ILOM based systems.

Scope

Details

     The following definitions are needed to review this document:

• RTC (Real Time Clock) - the battery backed clock that maintains time when the system is powered down.
• TOD (Time of Day) - the clock maintained by an operating system ILOM/Solaris.

Some systems designed prior the Sun Fire T2000 named the RTC within the NVRAM/IDPROM as the "TOD clock" since those systems only had one Solaris instance.  These definitions could be confusing to long term SPARC system users.

     Initialization of the clocks in SPARC systems with ILOMs has evolved greatly from those used in the older SPARC 3i systems like the Sun Fire V240, due to the addition of LDoms.  The SPARC 3i systems only had one instance of Solaris that could run on the system, so the Solaris TOD clock would directly initialize from the battery backed-up RTC after making Timezone modifications.  Each LDom of the newer systems may use differing Timezones or may sync to different NTP servers, so there is no direct correlation between the RTC & each instance of the Solaris clocks. 

    Please note that it is always best to have both the Solaris host(s) & the ILOM configured to use NTP.  Also, the user should immediately set the RTC via the ILOM after the Motherboard or Service Processor are replaced & prior Solaris boot.

    This evolution has provided differing methods of ILOM based systems as follows:

T5xx0 servers

    The ILOM maintains the TOD for each Solaris instance, so NTP updates to the Solaris host have no affect on the TOD.

    The ILOM will store the offset from the RTC's time for each Solaris instance.  During firmware upgrades, user's should choose Preserve Existing Configuration option so that the Solaris offsets are not affected.  Commands to set the ILOM's clock will also set the RTC to the same value, but then the clocks will start to drift from that point.

    During Solaris boot after power on, the offset will be added to the time of the RTC & be used by Solaris to initialize it's time.  A Solaris init 6 will not reload it's time from the RTC/offset.  Solaris will control it's own time after boot & not affect the RTC's time.

    When a user set's the ILOM time, the Solaris offsets are modified so that the Solaris time is not changed.  If a user advances the ILOM clock by 1 hour, then each Solaris offset will be decreased by 1 hour so that the next Solaris boot time remains in synchronization with the current Solaris time.  Please note that it is unlikely that a user would modify the ILOM time if it is synchronized to an NTP server. 

    The Solaris offsets will be modified if a user sets the Solaris time via the date command.  This will allow the new offset to synchronize the next boot time to the new time set in Solaris.  It is unlikely that a user will set Solaris time when it uses NTP.
    

T3-x (& later CMT servers)

    These servers contain two RTCs, one for the ILOM and one for the host/Solaris.  

1. The RTC on the ILOM side can be synced via NTP, but will have no effect on the host-side RTC.  
2. The host-side RTC's time is initialized in the factory to GMT and then never modified (if battery failure).

    Commands to set the ILOM's clock will also set it's RTC to the same value, but then both clocks will start to drift from that point.  The ILOM's NTP time corrections were not moved to the ILOM RTC by ILOM firmware prior version 3.2.1.  Thus, the RTC's drift would not be corrected by NTP for the future reboot of each Solaris instance.  Firmware 3.2.1 on the T3-x (& newer platforms) provides an hourly update of the RTC from the ILOM's clock to obtain NTP correction.  Again, both clocks will drift for an hour until the next hourly update is made.  Please note that firmware 3.2.4 will not perform the hourly ILOM clock to ILOM RTC time updates unless the ILOM is using NTP since the accuracy of the RTC's clock is better than the ILOM's clock.

    On power up, the host RTC's time is loaded into the FPGA counter, fpga_tod_counter for use as the time base for all LDoms.

    The host's control domain has a tod_offset which is stored in NVRAM on the host's RTC so is persistent over power cycles.  This tod_offset is modified for any changes to control-domain time (i.e. NTP or the date cmd).  It is also cleared to zero at boot time if the last-booted-config is not the same as currently-booting-config, if the RTC's battery is low while power is off, or if the magic word in the RTC is incorrect (i.e. NVRAM corrupt). 

    The tod_offsets for guest LDoms are stored as part of the LDom configuration saved to the SP.  Changes to the tod_offsets while the host is running are tracked by the Hypervisor, but those changes do not persist across a powercycle, unless the user manually re-saves his LDom configuration.  Clock updates (i.e. NTP, date) in any guest will result in an update of the respective tod_offset value.  This value will be relative to the fpga_tod_counter, which may drift relative to both real time (e.g. an NTP reference) as well as the RTC time.

If high Solaris boot time accuracy is required, then it is highly recommended that the Solaris NTP be configured to quickly synchronize the time.  There are two NTP services/versions available in Solaris which must be taken into account to determine which configuration changes are needed:

NTPv3 xntpd  available in Solaris 8, 9, and 10
NTPv4  ntpd  available in Solaris 10 Update 8 10/09 Release and later, and Solaris 11 all Releases.

In Solaris 10 the NTPv3 SMF Service is ntp that starts/usr/lib/inet/xntpd.
In Solaris 10 Update 8 and later, the NTPv4 SMF Service is ntp4 that starts /usr/lib/inet/ntpd.
In Solaris 11, all Releases, the NTPv4 SMF Service is ntp that starts /usr/lib/inet/ntpd.
In Solaris 11.3 SRU2.4 or greater, set SMF ntp Service Property config/allow_step_at_boot to true to allow time steps at boot (See doc 2087255.1 ).

The NTPv3 ntp Service executes the ntpdate command before xntpd executed, when the service is enabled.  ntpdate will immediately step the clock to be within sync range with the configured NTP server(s).  No configuration changes are necessary when using NTPv3 ntp Service.

The NTPv4 Service does not execute ntpdate when the service is enabled.  The ntpd daemon requires 5 good polls from at least one of the configured NTP servers before it will make a decision and chose a server for synchronization.  The initial synchronization could take 5 minutes or more.  To alter this behavior the burst and iburst options can be added to the server options in the /etc/inet/ntp.conf file

server  10.x.x.x burst iburst
server  10.x.x.x burst iburst
server  10.x.x.x burst iburst
server  10.x.x.x burst iburst
server  10.x.x.x burst iburst

These will send a burst of six or eight packets depending on the server being reachable or unreachable, and removes the delay between packets being sent which results in a faster initial synchronization.  Refer to the ntp.conf(4) man page for more details about these options.

Please note that the system's local clock entry, server 127.127.1.0, should not be set in the ntp.conf when concerned about time accuracy on boot.  The boot time could be in error for several minutes until one of the NTP servers becomes trusted.

This solution has no attachment