Description

Use of the (default) ILOM "Performance" Power Management Policy may result in significant impairment of performance of Solaris 11 Domains (Control and Guest) relative to the "Disabled" policy for Solaris 11 Operating Systems on M5 and T5 systems. The performance impairment may happen regardless of whether the control domain is running Solaris 10 or Solaris 11.

Occurrence

This issue can occur on Solaris 11 domains, both control and guest, on M5 and T5 platforms with the following control/primary domains:

SPARC Platform

• Solaris 11.1 through Solaris 11.1.9.5.0

Notes:

1. This issue only affects Solaris 11 domains, both control and guest, on the M5 and T5 (SPARC) platforms that are using the default ILOM Power Management "Performance" policy.

    To determine whether a system is using this default "Performance" policy, log into the Service Processor and execute the following:

      -> show /SP/powermgmt policy

         /SP/powermgmt
          Properties:
            policy = performance

    If the policy returned is "performance", then the system is using the Performance policy.  If anything else is returned, then the system is not using the Performance policy.

2.  This issue does not affect Solaris 10 or earlier, or any platform other than the M5 and T5 SPARC platforms.

Symptoms

To determine whether a system is experiencing this impaired performance, compare set workloads with the ILOM power management policy set to "disabled" vs. the default "performance" setting.  If there is more than a negligible difference in performance, the policy is not behaving as expected.

Workaround

To restore the expected behavior in this policy, manually correct the power management settings as shown below on all domains (both control and guest) affected by this impaired performance:

      # poweradm set time-to-full-capacity=2000
      # poweradm set time-to-minimum-responsiveness=0
      # poweradm set administrative-authority=smf

This manually implements the power management settings to be consistent with the forthcoming resolution for this issue.  These settings are persistent across reboots. Once the fix for this issue has been installed, the 'administrative-authority' setting should be returned to "platform".

Resolution

This issue is addressed in the following release, which must be applied to the control domain(s) of the M5 and T5 platforms:

SPARC Platform

• Solaris 11.1.10.5.0 or later

History

26-Aug-2013: Document released; State: Resolved
04-Dec-2017: Updated for clarification and maintenance, no change in content

Background: We have observed a number of substantial performance problems on T5/M5/M6 domains
running in Performance power management policy.  These problems are not observed on T4 systems.
The issue is that in Performance (and Elastic) policy, the power aware dispatcher in Solaris places
processors into the slowest pstate, which is 1/36 the clock frequency of the fastest pstate.  Compounding
this, the latency for returning to the fastest state is about 14 ms.  The result is that under loads with
certain characteristics, the processor spends a significant amount of time running these loads at a slow
processor speed before the processor returns to full speed.

Why is this not seen on T4?  T4's slowest state is 1/2 the performance of the fastest, and the latency to
resume full speed is much less (1.2 ms).  Therefore, the processor is never running excessively slowly,
and it returns to full performance much more expeditiously.

The nature of the fix is that in performance policy on M5/T5, we will limit the lowest performance
state to one that can return to full performance within a much shorter period of time, and also to
limit the lowest frequency to one with higher performance.  We are still collecting data to support
the exact tuning parameters.  It is a low risk change that will just change one variable (the ttfc
for Performance policy) that is calculated at startup.

We consider this fix sufficiently urgent to warrant a patch because by default the system runs in
Performance policy.  Performance policy is documented as having minimal impact on system performance,
which we have determined is not true in some significant cases.

The power settings given in the workaround are the values that will be delivered as the fix for this
bug 17179054.  As such, the workaround delivers the same change as the formal fix will deliver for this
issue.  The formal fix is in LDoms, to limit the time-to-full-capacity on as described in the workaround.

This performance issue does not affect S11 FCS or S10 domains, because they do not have the power-aware
dispatcher (ER 15620847), which was integrated in S11.1.

Questions regarding this document should be addressed to
sunalertpublication_us_grp@oracle.com and copy the
submitter/responsible engineer listed below.

Comments:

Internal Contributor/Submitter: robert.krawitz@oracle.com
Internal Eng Responsible Engineer: julia.harper@oracle.com
Internal Services Knowledge Analyst: david.mariotto@oracle.com
Internal Eng Business Unit Group: Solaris RPE

References

This solution has no attachment