Exalogic Patch Set Update (PSU) Release 2.0.6.2.2 (Solaris

Asset ID:	1-79-2027107.1
Update Date:	2016-06-23
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 2027107.1 : Exalogic Patch Set Update (PSU) Release 2.0.6.2.2 (Solaris - Physical) for July 2015

Applies to:

Exalogic Elastic Cloud X5-2 Hardware
Oracle Exalogic Elastic Cloud Software - Version 2.0.4.0.0 to 2.0.6.2.1
Exalogic Elastic Cloud X3-2 Hardware
Oracle Exalogic Elastic Cloud X2-2 Hardware
Exalogic Elastic Cloud X4-2 Hardware
Oracle Solaris on x86-64 (64-bit)

Purpose

Oracle Exalogic is an integrated hardware and software system designed to provide a complete platform for a wide range of application types and widely varied workloads. It combines optimized Oracle Fusion Middleware software like WebLogic server, JRockit, Coherence with industry-standard Sun Server and Storage hardware and InfiniBand networking. The purpose of this document is to provide specific information around July 2015 Patch Set Update (PSU) for that system.

Scope

The target audience of this document are engineers and system administrators who plan to apply the Exalogic PSU.

This document provides the following:

README files that are included in the July 2015 PSU for convenience. These can be used for reference without having to download the PSU first.
Known issues with the PSU - these provide a list of known issues along with workarounds/solutions, if available. Engineers and administrators planning to apply the PSU should ensure to review these before applying the PSU.

This document will be kept up-to-date with updates to errata and known issues.

Details

Note: This PSU includes updates to ELLC 14.2.3

Patch Download

Released: July 2015

Product Version: 2.0.6.2.2 (on X2-2/X3-2/X4-2/X5-2) for Oracle Solaris x86-64

PATCH 20860046 - EXALOGIC SOLARIS 2.0.6.2.2 (on X2-2/X3-2/X4-2/X5-2) PATCH SET UPDATE (PSU) FOR JULY 2015

Patch Readme Documentation

Refer to the readme documentation attached 20860046-Solaris.zip to this Document on how to upgrade the Exalogic infrastructure:

Exalogic X2-2,X3-2,X4-2 and X5-2 Systems: Upgrading to 2.0.6.2.2

The readme content structure layout of the 20860046-Solaris.zip:

20860046-Solaris
|
| - README.html
| - exalogic-lctools-14.2.3-release-notes.txt
| - docs/README.html
| - docs/components/ILOM_ComputeNode_Upgrade.html
| - docs/components/Index.html
| - docs/components/Menu.html
| - docs/components/NM2_Upgrade.html
| - docs/components/ZFS_Upgrade.html

INTERNAL

Pre-requisites

PCIe Renumbering check on ZS3-ES storage heads in Exalogic x4-2 and x5-2 racks

Refer to following Note which has information on this important pre-requisite check.

<Note 2087741.1>: Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks

Appendices

Appendix A: Fixed Bugs List

Please review Document ID : 2027076.1 - Exalogic Infrastructure July 2015 PSU - Fixed Bugs List.

Appendix B: Patching Known Issues

CheckHWnFWProfile fails on X5-2 Solaris.

Symptoms

On Physical Solaris X5-2 systems /opt/exalogic.tools/tools/CheckHWnFWProfile fails with:-

*******************************************************************************************************************

Does not contain supported disk controller. Required LSI MR9261-8i. But has LSI
SUN
SUN
SUN
SUN
SUN
SUN
SUN MR9261-8i
ZFS Storage 7320
ZFS Storage 7320
ZFS Storage 7320
ZFS Storage 7320
ZFS Storage 7320
ZFS Storage 7320
ZFS Storage 7320

*******************************************************************************************************************

Cause

This is due to a known issue that is tracked through the following bug:

BUG 20745068 - CHECKHWNFWPROFILE FAILS POST COMPUTE NODE UPGRADE TO APR15 PSU ON X5-2

There are two underlying issues that cause CheckHWnFWProfile to fail on Solaris X5-2 systems:

1. The supported configuration metadata is incorrect and contains an incorrect reference to the LSI MR9261-8i disk controller. The profile check against the actual version installed on the node (LSI MR9361-8i) fails. The supported version is LSI MR9361-8i; this error may be safely ignored

2. The MegaCli utility that is installed on X5-2 nodes is unable to detect the disk controller to obtain/upgrade/downgrade the installed firmware version. You may safely continue to use the node with the version of the disk controller firmware shipped from the factory.

Solution/Workaround

This error may be safely ignored and you may safely continue to use the node with the version of the disk controller firmware shipped from the factory.

CheckHWnFWProfile fails on Solaris with Zones over Shared Storage (ZOSS) over iSCSI devices.

Symptoms

On Physical Solaris systems with Zones over Shared Storage (ZOSS) over iSCSI devices

/opt/exalogic.tools/tools/CheckHWnFWProfile fails with:-

*******************************************************************************************************************

Does not contain supported disk controller. Required LSI MR9261-8i. But has
LSI SUN MR9361-8i

*******************************************************************************************************************

Cause

This is due to a known issue that is tracked through the following bug:

BUG 16634965 - CHECKHWNFWPROFILE FAILS DISK CONTROLLER CHECK AFTER APPLYING ISCSI SUPPORT PATCH

The underlying issue that cause CheckHWnFWProfile to fail on Solaris system is:

In addition to the PCI device (LSI MR9261-8i) that the check is looking for, additional iSCSI devices are configured for each zone created. The SCSI devices break the disk controller check logic implemented in the CheckHWnFWProfile profile check.

Solution/Workaround

This error may be safely ignored and does not affect any functionality. You may safely proceed with the PSU application using exapatch --force option.

Post Patching Switches, NFS mount point is in hung state

Symptoms:

During the NM2-GW upgrade, if the NFS servers stops responding, the patch upgrade process launched from the compute node may hang.

Cause:

Observed to happen while upgrade from PSU 2.0.4.0.6 or before (NM2 v2.1.4-1 or earlier). See bug 18465204

Solution/Workaround:

In such situation, force kill the upgrade process, restart the network services and retry the patching command. To restart network services, run:

# svcadm restart network/physical:default

Continuous errors like BIND() FAILED, ERRNO in OSWatcher service Log

Symptoms:

On Physical Solaris systems, in the OSWatcher service log(/var/svc/log/system-oswatcher:default.log) continuous errors(listed below) are observed:-

default-info: Unable get statistics: Bad address

bind() failed, errno: 14 (Bad address)

Cause:

In the Physical Solaris systems rds-info and rds-ping commands are returning these above listed error messages.

ExaLogic Solaris doesn't use rds and the rds service is disabled.

See bug 21293201

Solution/Workaround:

As part of ExaLogic Physical Solaris July'15 PSU, Oswatcher service is disabled. No action is required after applying ExaLogic Physical Solaris July'15 PSU.

ZFS Upgrade Failed. Storage node ilom console indicates error in joining cluster.

Symptoms:

While applying the July 2015 PSU on X5-2 rack, the upgrade of ZS3-ES may fail. The exapatch fails with:-

"ERROR: Could not ssh to ZFS host xx.xx.xx.xx after upgrade. "

Further,

When I see the storage node ilom of the failed head, it is stuck at this stage :

Oracle ZFS Storage ZS3-ES Configuration
    Copyright (c) 2008, 2015, Oracle and/or its affiliates. All rights
reserved.

    NET-0 <=>  NET-1 <=>  NET-2 <=>  NET-3 <X>

    SUNW-MSG-ID: AK-8000-2U, TYPE: Defect, VER: 1, SEVERITY: Major
    EVENT-TIME: Tue Jun  2 17:01:26 UTC 2015
    PLATFORM: i86pc, CSN: 1427NML0E5, HOSTNAME: scat01sn02
    SOURCE: aktty, REV: 1.0
    EVENT-ID: 1fc05eee-db80-e94d-d6c2-c06714389111
    DESC: The appliance experienced an unrecoverable protocol error while
    attempting to join a cluster of appliances.  Refer to
    http://sun.com/msg/AK-8000-2U for more information.
    AUTO-RESPONSE: No automated response is possible for this defect.
    IMPACT: The appliance cannot be configured into the cluster.
    REC-ACTION: Reboot the system and attempt to configure the cluster again.
    Contact your vendor for support.

    ESC-3: Halt   ESC-4: Reboot   ESC-5: Info

    For help, see http://www.oracle.com/goto/ZS3-ES/docs/

Cause:

@ Details of this issue are in this MOS note:
@ .
@ Oracle ZFS Storage Appliance: ZS3-X Clustron Card in PCIE Slot 2 Does Not
@ Initialize During boot. (Doc ID 1910623.1)
@ .
@ The following bug is referenced from the MOS note:
@ .
@ BUG 18672251 - AK-8000-2U: unrecoverable protocol error on the ZS3-ES
@ (X4170M3) appliance

Solution/Workaround:

Manually reboot the storage head with ESC-4. Refer to <Note 1910623.1>.

PSU Upgrade fails while upgrading ZFSSA.

Symptoms: While upgrading ZFS software from 2011.x to 2013.x using exapatch, the upgrade fails after upgrading one of the two storage control heads.

The following error is observed in the exapatch logs:

COMPONENT DIAGNOSED TYPE DESCRIPTION
problem-000 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS1025TM6Z/a'
in pool 'exalogic' failed.

problem-001 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS1035TM6Z/a'
in pool 'exalogic' failed.

problem-002 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS102FTM6Z/a'
in pool 'exalogic' failed.

problem-003 2015-5-1 17:45:07 Major Fault ZFS device

Pre-patch check fails on the second storage head causing exapatch to exit the upgrade operation.

Cause: The pre-patch check failure is due to a known issue tracked through the following ZFS bug:

BUG 19024367 - Peer node's RZ listed under problems and new problem entry is created on repair

- Readzilla belonging to peer node are marked faulty and listed under
problems
- When an attempt is made to repair the unavailable readzilla entry, one more problem
entry gets added

A fix for the above bug is not yet available for Exalogic. Implement the following workaround to proceed with the ZFS upgrade.

Solution/Workaround:

Since Exapatch does not allow you to proceed, the software on the of the remaining head must be upgraded manually.

The following Exalogic patch contains a manual procedure for the ZFS software upgrade

https://updates.oracle.com/download/20983458.html

Upgrade from EECS 2.0.4.0.4 and earlier releases (containing NM2 FW v2.0.x) to EECS 2.0.4.0.5 and newer releases (containing NM2 FW v2.1.x) requires downtime to perform the switch FW upgrade.

For information on this known issue refer to Exalogic Infrastructure PSU Upgrade Known Issues Note 1571367.1, section "Upgrade from EECS 2.0.4.0.4 and earlier releases (containing NM2 FW v2.0.x) to EECS 2.0.4.0.5 and newer releases (containing NM2 FW v2.1.x) requires downtime to perform the switch FW upgrade".

_____________________________________________________________________________________________________________________________________________________________________________________

Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Refer to <Note 2114516.1> for details on this known issue.

______________________________________________________________________________________________________________________________________________________

Appendix C: FAQs

Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching

Refer to <Note 1997466.1> for details

References

<NOTE:1329262.1> - How to Perform a Healthcheck on Exalogic
<NOTE:1449226.1> - Exachk Health-Check Tool for Exalogic
<NOTE:2087741.1> - Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks
<NOTE:1314535.1> - Exalogic Patch Set Updates (PSU) Master Note
<NOTE:1997466.1> - Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching
<NOTE:2114516.1> - Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Attachments

This solution has no attachment