Exalogic Patch Set Update (PSU) Release 2.0.6.2.1 (Solaris

Asset ID:	1-79-1995235.1
Update Date:	2016-06-23
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 1995235.1 : Exalogic Patch Set Update (PSU) Release 2.0.6.2.1 (Solaris - Physical) for April 2015

Applies to:

Exalogic Elastic Cloud X3-2 Hardware - Version X3 to X3 [Release X3]
Oracle Exalogic Elastic Cloud Software - Version 2.0.4.0.0 to 2.0.6.2.0
Exalogic Elastic Cloud X4-2 Hardware - Version X4 to X4 [Release X4]
Oracle Exalogic Elastic Cloud X2-2 Hardware - Version X2 to X2 [Release X2]
Exalogic Elastic Cloud X5-2 Hardware - Version X5 to X5 [Release X5]
Oracle Solaris on x86-64 (64-bit)

Purpose

Oracle Exalogic is an integrated hardware and software system designed to provide a complete platform for a wide range of application types and widely varied workloads. It combines optimized Oracle Fusion Middleware software like WebLogic server, JRockit, Coherence with industry-standard Sun Server and Storage hardware and InfiniBand networking. The purpose of this document is to provide specific information around April 2015 Patch Set Update (PSU) for that system.

Scope

The target audience of this document are engineers and system administrators who plan to apply the Exalogic PSU.

This document provides the following:

README files that are included in the April 2015 PSU for convenience. These can be used for reference without having to download the PSU first.
Known issues with the PSU - these provide a list of known issues along with workarounds/solutions, if available. Engineers and administrators planning to apply the PSU should ensure to review these before applying the PSU.

This document will be kept up-to-date with updates to errata and known issues.

Details

Note: This PSU includes updates to ELLC 14.2.2

Patch Download

Released: April 2015

Product Version: 2.0.6.2.1 (on X2-2/X3-2/X4-2/X5-2) for Oracle Solaris x86-64

PATCH 20448180 - EXALOGIC SOLARIS 2.0.6.2.1 (on X2-2/X3-2/X4-2/X5-2) PATCH SET UPDATE (PSU) FOR APRIL 2015

Patch Readme Documentation

Refer to the readme documentation attached 20448180-Solaris.zip to this Document on how to upgrade the Exalogic infrastructure:

Exalogic X2-2,X3-2,X4-2 and X5-2 Systems: Upgrading to 2.0.6.2.1

The readme content structure layout of the 20448180-Solaris.zip:

20448180-Solaris
|
| - README.html
| - exalogic-lctools-14.2.2-release-notes.txt
| - docs/README.html
| - docs/components/ILOM_ComputeNode_Upgrade.html
| - docs/components/Index.html
| - docs/components/Menu.html
| - docs/components/NM2_Upgrade.html

Known Issue with data corruption in ZFS Storage Appliance Software (2013.1.3.5) bundled in April 2015 PSU

There is a known issue with data corruption in the ZFS version (2013.1.3.5) bundled in the April 2015 PSU. Please refer to the following MOS Note for details:

<Note 2006121.1>: Exalogic: Risk of Data Corruption in ZFS Storage Appliance Software (2013.1.3.5) Bundled in April 2015 PSU

All applications of April 2015 PSU MUST apply the updated ZFS version provided through patch 20983458 as a replacement to the version included in the PSU.

INTERNAL

Pre-requisites

PCIe Renumbering check on ZS3-ES storage heads in Exalogic x4-2 and x5-2 racks

Refer to following Note which has information on this important pre-requisite check.

<Note 2087741.1>: Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks

Appendices

Appendix A: Fixed Bugs List

Please review Document ID : 1995246.1 - Exalogic Infrastructure April 2015 PSU - Fixed Bugs List.

Appendix B: Patching Known Issues

During the April 2015 PSU patching, compute nodes will crash if the IB master subnet manager is not running on an NM2-GW switch on the Exalogic rack

Symptoms

When patching IB switches on an Exalogic rack, some or all compute nodes lose IB communication capabilities. This causes them to reboot.

Cause

The problem is that when an IB master subnet manager's host IB switch is being upgraded it needs to reboot. Therefore, before the upgrade, the subnet manager daemon on the NM2-GW switch is disabled. The other IB components on the rack running the subnet manager daemon negotiate a new master subnet manager. If the new master subnet manager ends up on a device other than an NM2-GW switch on the Exalogic rack, compute nodes will lose their IB connectivity and then crash due to the fact that they cannot communicate with the cluster.

Solution/Workaround

The workaround is to follow the solution documented in the Document ID : 1682501.1 - Setting up the subnet manager in a multirack configuration containing Exalogic/BDA and Exadata/SSC/Expansion Rack.

Note that the configuration must be correct on ALL IB subnet managers on the IB fabric, and is not limited to the IB switches on the Exalogic rack.

Upgrading NM2 Switch Firmware on full rack Exalogic systems or multi-rack Exa systems causing Network outage

NOTE:

The issue does not occur when Exalogic racks (one eighth, quarter, half ) are connected to Exadata racks.

Symptoms

Network outages on the InfiniBand fiber.

Cause

The NM2 Port ID space has increased four times from firmware version 2.0 to 2.1. Therefore, port ID collisions may occur if NM2 switches with firmware version 2.0 and 2.1 are run on the same fabric. This issue can cause network outages on the InfiniBand fiber.

Solution/Workaround

In order to prevent this issue, manipulation of the switch 'GWInstance' value may be required to ensure unique port IDs are generated by performing the following steps:

Log in to each NM2-GW switch as root and run:

# showgwconfig

Note the GWInstance "Running Value" on each of the switches.

1. Ensure that all the GWInstance values are even numbers
2. Ensure that GWInstance values when multiplied by 4, are greater than the largest GWInstance value.

To change a GWInstance value to 16, run:

[root@nm2gw-ib01 ~]# setgwinstance 16
Stopping Bridge Manager.. [ OK ]
Starting Bridge Manager. [ OK ]
root@nm2gw-ib01 ~]#

For example, if the GWInstance values of four NM2 switches are 10, 20, 30 & 40, it will be necessary to change the GWInstance value of 10 to 16.

Upgrade from EECS 2.0.4.0.4 and earlier releases (containing NM2 FW v2.0.x) to EECS 2.0.4.0.5 and newer releases (containing NM2 FW v2.1.x) requires downtime to perform the switch FW upgrade.

For information on this known issue refer to Exalogic Infrastructure PSU Upgrade Known Issues Note 1571367.1, section "Upgrade from EECS 2.0.4.0.4 and earlier releases (containing NM2 FW v2.0.x) to EECS 2.0.4.0.5 and newer releases (containing NM2 FW v2.1.x) requires downtime to perform the switch FW upgrade".

CheckHWnFWProfile fails on X5-2 Solaris.

Symptoms

On Physical Solaris X5-2 systems /opt/exalogic.tools/tools/CheckHWnFWProfile fails with:-

*******************************************************************************************************************

Does not contain supported disk controller. Required LSI MR9261-8i. But has
LSI MR9361-8i

*******************************************************************************************************************

Cause

This is due to a known issue that is tracked through the following bug:

BUG 20745068 - CHECKHWNFWPROFILE FAILS POST COMPUTE NODE UPGRADE TO APR15 PSU ON X5-2

There are two underlying issues that cause CheckHWnFWProfile to fail on Solaris X5-2 systems:

1. The supported configuration metadata is incorrect and contains an incorrect reference to the LSI MR9261-8i disk controller. The profile check against the actual version installed on the node (LSI MR9361-8i) fails. The supported version is LSI MR9361-8i; this error may be safely ignored

2. The MegaCli utility that is installed on X5-2 nodes is unable to detect the disk controller to obtain/upgrade/downgrade the installed firmware version. You may safely continue to use the node with the version of the disk controller firmware shipped from the factory.

Solution/Workaround

This error may be safely ignored and you may safely continue to use the node with the version of the disk controller firmware shipped from the factory.

Post Patching Switches, NFS mount point is in hung state

Symptoms:

During the NM2-GW upgrade, if the NFS servers stops responding, the patch upgrade process launched from the compute node may hang.

Cause:

Observed to happen while upgrade from PSU 2.0.4.0.6 or before (NM2 v2.1.4-1 or earlier). See bug 18465204

Solution/Workaround:

In such situation, force kill the upgrade process, restart the network services and retry the patching command. To restart network services, run:

# svcadm restart network/physical:default

"/etc/inet/hosts" file on Exalogic Solaris Compute Nodes getting changed after applying Exalogic Solaris April 2015 PSU 2.0.6.2.1

Refer to <Note 2056597.1> for details on this known issue.

____________________________________________________________________________________________________________________________________________________________________________________

Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Refer to <Note 2114516.1> for details on this known issue.

Appendix C: Troubleshooting

ExaPatch prePatchCheck option fails with ERROR:FAILED - /opt/exalogic.tools/CheckHWnFWProfile.

Problem

ExaPatch prePatchCheck option fails with ERROR:

FAILED - /opt/exalogic.tools/tools/CheckHWnFWProfile

CheckHWnFWProfile may have failed due to valid upgraded firmware but /usr/lib/init-exalogic-node/supported_config file still references original firmware version. For e.g.

Supported disk controller Version: 12.12.0-0079

Current disk controller Version  : 12.12.0-0178

Disk controller is not at the supported firmware version. It requires firmware update...

Supported disk controller Firmware at: /opt/exalogic/firmware/disk_controller/12_12_0_0079.rom

Solution

After making sure there are no other errors, ignore this precheck and continue patching. When patching the compute node, please use the -f flag with Exapatch command line option.

For e.g., to override pre-patching checks while patching compute nodes, run the following command:

[root@compute-node2]# /exalogic-lctools/bin/exapatch -a patch cn -h <compute_node_ip> --force

Appendix D: FAQs

Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching

Refer to <Note 1997466.1> for details

References

<NOTE:1329262.1> - How to Perform a Healthcheck on Exalogic
<NOTE:1449226.1> - Exachk Health-Check Tool for Exalogic
<NOTE:2087741.1> - Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks
<NOTE:1997466.1> - Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching
<NOTE:2006121.1> - Exalogic: Risk of Data Corruption in ZFS Storage Appliance Software (2013.1.3.5) Bundled in April 2015 PSU
<NOTE:2056597.1> - "/etc/inet/hosts" File On Exalogic Solaris Compute Nodes Getting Changed After Applying Exalogic Solaris April 2015 PSU 2.0.6.2.1
<NOTE:2114516.1> - Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade
<NOTE:1314535.1> - Exalogic Patch Set Updates (PSU) Master Note

Attachments

This solution has no attachment