Sun Fire[TM] 12K/15K/20K/25K: Clock Management

Asset ID:	1-71-1003344.1
Update Date:	2017-07-10
Keywords:

Solution Type Technical Instruction Sure

Solution 1003344.1 : Sun Fire[TM] 12K/15K/20K/25K: Clock Management

Applies to:

Sun Fire 12K Server - Version All Versions and later
Sun Fire 15K Server - Version All Versions and later
Sun Fire E20K Server - Version All Versions and later
Sun Fire E25K Server - Version All Versions and later
All Platforms

Goal

The purpose of this document is to describe SMS clock management in the 12K/15K/20K/25.

Solution

Each SC has several clock frequency generators to source clocking for the entire platform. The generators are selectable and programmable via software, including the ability to enable one SC to mirror the other's clock signal (REF_SEL bit in the Gchip). When both SCs generate identical clock signals, this is called phase lock.

On every Expander, Slot 0, Slot 1, and Centerplane Support Board exists a Smart Phased Lock Loop (SPLL) circuit that accepts two clock inputs, one from each System Controller. The SPLL selects and uses one of the inputs to generate the local clock fanout for its respective board. The circuit has the ability to dynamically switch to the alternate clock source without interrupting the local clock fanout (assuming phase lock). Such a switch does not require software intervention.

The SPLL is programmable and provides status via an I2C device. The bits important to this discussion are:

          SPLL BIT DEFINITIONS
          |Bit Type       | Type | POR | Description                          |
          +---------------+------+-----+--------------------------------------+
          | ALARM_RESET_L |  RW  |  1  | 0 resets the phase clock alarm       |
          |               |      |     | and aligns CLK_SELECTED with SEL_CLK |
          +---------------+------+-----+--------------------------------------+
          | SEL_CLK_L     |  RW  |  1  | 1 selects clock source 0 (SC0); 0    |
          |               |      |     | selects clock source 1 (SC1)         |
          +---------------+------+-----+--------------------------------------+
          | MAN_OVERRIDE  |  RW  |  1  | 1 disables SPLL internal select      |
          |               |      |     | circuitry                            |
          +---------------+------+-----+--------------------------------------+
          | CLK_SELECTED  |  RO  |     | Indicates which clock source is      |
          |               |      |     | currently selected, 0 for SC0, 1 for |
          |               |      |     | SC1.                                 |
          +---------------+------+-----+--------------------------------------+
          | INPUT1BAD     |  RO  |     | 1 indicates clock 1 (SC1) is bad     |
          |               |      |     | relative to the feedback signal      |
          +---------------+------+-----+--------------------------------------+
          | INPUT0BAD     |  RO  |     | 1 indicates clock 0 (SC0) is bad     |
          |               |      |     | relative to the feedback signal      |
          +---------------+------+-----+--------------------------------------+

Beginning with SMS 1.2 patch level 112481-06, there is much more monitoring and control performed by SMS in the clock management arena. As an overall guideline, it is attempted to set a component's selected clock source to the MAIN SC. But, more generally, software manages and monitors component clock status using the following rules:

hwad programs REF_SEL

When an SC enters the SPARE role, REF_SEL is cleared in the Gchip.

When REF_SEL is 0, frequency tracking is enabled so the SPARE SC's clock mirrors the MAIN and phase lock is achieved.

hwad localizes clocks to the MAIN SC

When an SC becomes MAIN, if the SC clocks are phase locked, hwad programs SEL_CLK_L in all SPLLs to select the MAIN SC's clock input.
If a component's SPLL indicates the input from the MAIN SC is bad, SEL_CLK_L is not changed.

If the SC clocks are not phase locked, no changes are made to any component's SEL_CLK_L.

At poweroff of non-MAIN SC, localize clocks to the MAIN SC

When an SC is powered off, it will cease to provide clocking. Any SPLLs with CLK_SELECTED matching the SC being powered off are attempted to be reprogrammed. If the SPLL in question does not have a bad input for the MAIN SC and phase lock is active, SEL_CLK_L is changed to select the MAIN SC. Otherwise, a warning message is printed informing the user of a possible loss of clock to an active component.

User intervention is required to force the poweroff.

At component power on, localize clocks to the MAIN SC

When a component is first powered on, SEL_CLK_L is programmed to select the MAIN SC.

esmd monitoring

esmd polls the status of the INPUT0BAD and INPUT1BAD in all SPLLs periodically. If a bad input is detected, MAN_OVERRIDE is set in that SPLL to disable automatic switching. Appropriate messages are logged.

esmd also monitors phase lock. If phase lock is lost, MAN_OVERRIDE is set in all SPLLs. No clock source selections are changed. Likewise, if phase lock is active, MAN_OVERRIDE is cleared in all SPLLs (excepting those with bad inputs).

These rules do not preclude the possibility of domain interruptions due to clock failures, but all attempts are made to avoid such an event. It is therefore important to monitor the platform message logs for esmd messages noting bad inputs on components and addressing them as soon as possible.

As of SMS 1.4 the command showboards -c shows detailed information about the clock status per component. For SMS version before 1.4 the alternative is the showclocksrc utility.

Background Information:

Sample output of showboards -c :

v4u-15kb-sc0:sms-svc:1> showboards -c
Retrieving board information. Please wait.
...........................
                     Current        SC0 Clock     SC1 Clock    Auto-Clock
Location    Pwr    Clock Source       Status        Status     Selection
--------    ---    ------------     ----------    ----------   ----------
CS0         On      SC0 Clock          Good          Good       Enabled
CS1         On      SC0 Clock          Good          Good       Enabled
EX0         On      SC0 Clock          Good          Good       Enabled
EX1         On      SC0 Clock          Good          Good       Enabled
EX2         On      SC0 Clock          Good          Good       Enabled
...
EX15        On      SC0 Clock          Good          Good       Enabled
EX16        On      SC0 Clock          Good          Good       Enabled
EX17        On      SC0 Clock          Good          Good       Enabled
SB0         On      SC0 Clock          Good          Good       Enabled
SB1         On      SC0 Clock          Good          Good       Enabled
SB2         On      SC0 Clock          Good          Good       Enabled
...
SB6         Off          -              -             -            -
...
SB15        On      SC0 Clock          Good          Good       Enabled
SB16        On      SC0 Clock          Good          Good       Enabled
SB17        On      SC0 Clock          Good          Good       Enabled
IO0         On      SC0 Clock          Good          Good       Enabled
IO1         On      SC0 Clock          Good          Good       Enabled
IO2          -           -              -             -            -
IO3         On      SC0 Clock          Good          Good       Enabled
IO4         On      SC0 Clock          Good          Good       Enabled
IO5         Off          -              -             -            -
...
IO15        On      SC0 Clock          Good          Good       Enabled
IO16        On      SC0 Clock          Good          Good       Enabled
IO17        On      SC0 Clock          Good          Good       Enabled

Keywords
15K, 12K, SF15K, SF12K, Sun Fire 15K, Enterprise, Server, Sun Fire 12K, clock, showclocksrc, SPLL

Internal Section

Examples of esmd messages at http://panacea.uk.oracle.com/twiki/bin/view/Products/SCshowclocksrc

Beginning with SMS1.4, showclocksrc is deprecated. In SMS1.4 and later, use 'showboards -c' to report clock information. Refer to RFE 4617511

Attachments

This solution has no attachment