Description

Certain firmware versions (as listed below) on SPARC T7/M7/S7 Systems may cause applications using Data Analytics Accelerator (DAX), such as Oracle Database 12c and other application leveraging the OpenDAX API, to hang if POST (Power On Self Test) is run prior to the Operating System (OS) booting. When this occurs, attempts to recover the application by rebooting the OS will cause the system to hang during the boot process.

Occurrence

This issue can occur on the following platforms:

SPARC Platform

• SPARC T7-1 with Firmware version 9.7.1.c (as delivered in patch 23291631)
• SPARC T7-2 with Firmware version 9.7.1.c (as delivered in patch 23291632)
• SPARC T7-4 with Firmware version 9.7.1.c (as delivered in patch 23291633)
• SPARC M7-8 with Firmware version 9.7.1.b (as delivered in patch 23291634)
• SPARC M7-16 with Firmware version 9.7.1.b (as delivered in patch 23291634)
• SPARC S7-2 with Firmware version 9.7.2.c (as delivered in patch 23632951)
• SPARC S7-2L with Firmware version 9.7.2.c (as delivered in patch 23632952)
• SPARC Netra S7-2 with Firmware version 9.7.2.c (as delivered in patch 23632954)

To determine the firmware version installed on the system, use the following ILOM command:

    -> show /HOST sysfw_version

Symptoms

When this issue occurs and the affected application hangs, there are no predictable symptoms that would indicate the described issue has occurred, as the response is dependent upon the application affected.

If the OS is rebooted, the boot process will hang as in the example shown below:

    sca-mxx-0xx-pd0 console login: root
    Password:
    Last login: Fri Jun 10 06:40:10 2016 on console
    Jun 10 07:38:41 sca-mxx-0xx-pd0 login: ROOT LOGIN /dev/console

    Oracle Corporation SunOS 5.11 11.3 March 2016
    You have new mail.
    root@sca-mxx-0xx-pd0:~#
    root@sca-mxx-0xx-pd0:~#
    root@sca-mxx-0xx-pd0:~# date
    Fri Jun 10 07:38:43 PDT 2016
    root@sca-mxx-0xx-pd0:~#
    root@sca-mxx-0xx-pd0:~#
    root@sca-mxx-0xx-pd0:~# uadmin 2 0
    syncing file systems... done

    Program terminated
    << ... system hang here ... >>

Workaround

To prevent this issue from occurring, either disable POST, or if POST was run, reboot the domain prior to starting any applications that use DAX as shown below:

To disable POST:

Set the host diag properties 'default_level’, 'hw_change_level’, and 'error_level' to 'off' as shown in the example below for a SPARC T7 system:

    -> set /HOST keyswitch_state=normal

    -> set /HOST/diag default_level=off
    -> set /HOST/diag hw_change_level=off
    -> set /HOST/diag error_level=off

Note: The 'keyswitch_state' property needs to be set to 'normal' before changing any of the host diag properties.

If POST was run during cold system boot or system level reset:

1. Do not configure or run any application that uses DAX.

2. Once the system completes POST and Solaris has booted, for systems with one or more Oracle VM Servers for SPARC Guest Domains running, select any one guest domain and reboot it. If the system does not run with any Oracle VM Server for SPARC Guest Domain, then reboot the OS one more time.

3. After the reboot completes, all applications using DAX can be manually started.

Note: This process must be repeated after every POST run.

If a system has experienced a hang as a result of this issue, a power cycle of the impacted HOST is required as shown in the following example for a SPARC T7 system:

    -> stop /HOST
    -> start /HOST

Resolution

This issue is addressed in the following releases:

SPARC Platform

• SPARC T7-1 with Firmware version 9.7.3.b (as delivered in patch 24483620)
• SPARC T7-2 with Firmware version 9.7.3.b (as delivered in patch 24483621)
• SPARC T7-4 with Firmware version 9.7.3.b (as delivered in patch 24483622)
• SPARC M7-8 with Firmware version 9.7.3.b (as delivered in patch 24483623)
• SPARC M7-16 with Firmware version 9.7.3.b (as delivered in patch 24483623)
• SPARC S7-2 with Firmware version 9.7.3.b (as delivered in patch 24483624)
• SPARC S7-2L with Firmware version 9.7.3.b (as delivered in patch 24483625)
• SPARC Netra S7-2 with Firmware version 9.7.3.b (as delivered in patch 24483627)

Patches

<Patch:24483620>
<Patch:24483621>
<Patch:24483622>
<Patch:24483623>
<Patch:24483624>
<Patch:24483625>
<Patch:24483627>

History

30-Sep-2016: Document released, status Workaround
06-Oct-2016: Updated with available firmware versions. State: Resolved

Internal Section: Comments:

This regression was caused by the putback for bug 22161598

Questions regarding this document should be addressed to
sunalertpublication_us_grp@oracle.com and copy the
submitter/responsible engineer listed below.

Internal Contributor/Submitter: marcel.widjaja@oracle.com
Internal Eng Responsible Engineer: shan.g.gao@oracle.com
Oracle Knowledge Analyst: jeff.folla@oracle.com
Internal Eng Business Unit Group: Systems RPE
Internal Associated SR IDs:
Internal Pending Patches:
Internal Resolution Patches:

References

This solution has no attachment