Exalogic Patch Set Update (PSU) Release 2.0.6.2.4 (Linux

Asset ID:	1-79-2093676.1
Update Date:	2016-06-23
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 2093676.1 : Exalogic Patch Set Update (PSU) Release 2.0.6.2.4 (Linux - Physical) for January 2016

Applies to:

Oracle Exalogic Elastic Cloud X2-2 Hardware - Version X2 to X2 [Release X2]
Exalogic Elastic Cloud X3-2 Hardware
Exalogic Elastic Cloud X4-2 Hardware - Version X4 to X4 [Release X4]
Oracle Exalogic Elastic Cloud Software - Version 2.0.0.0.0 to 2.0.6.2.4
Exalogic Elastic Cloud X5-2 Hardware
Linux x86-64

Purpose

Oracle Exalogic is an integrated hardware and software system designed to provide a complete platform for a wide range of application types and widely varied workloads. It combines optimized Oracle Fusion Middleware software like WebLogic server, JRockit, Coherence with industry-standard Sun Server and Storage hardware and InfiniBand networking. The purpose of this document is to provide specific information around January 2016 Patch Set Update (PSU) for that system.

Scope

The target audience of this document are engineers and system administrators who plan to apply the Exalogic PSU.

This document provides the following:

README files that are included in the January 2016 PSU for convenience. These can be used for reference without having to download the PSU first.
Known issues with the PSU - these provide a list of known issues along with workarounds/solutions, if available. Engineers and administrators planning to apply the PSU should ensure to review these before applying the PSU.

This document will be kept up-to-date with updates to errata and known issues.

Details

NOTE:

This PSU contains upgrades to both OL5, OL6 releases.
This PSU includes updates to ELLC 14.2.5

Patch Download

Released: January 2016

Product Version: 2.0.6.2.4 (on X2-2/X3-2/X4-2/X5-2) for Oracle Linux x86-64

PATCH 22136605 - EXALOGIC LINUX 2.0.6.2.4 (on X2-2/X3-2/X4-2/X5-2) PATCH SET UPDATE (PSU) FOR JANUARY 2016

Patch Readme Documentation

Refer to the readme documentation attached 22136605-Linux.zip to this Document on how to upgrade the Exalogic infrastructure:

Exalogic X2-2,X3-2,X4-2 and X5-2 Systems: Upgrading to 2.0.6.2.4

The readme content structure layout of the 22136605-Linux.zip:

22136601-Linux
|
| - README.html
| - exalogic-lctools-14.2.5-release-notes.txt
| - docs/README.html
| - docs/components/Enable_http_zfs.html
| - docs/components/ILOM_ComputeNode_Upgrade.html
| - docs/components/Index.html
| - docs/components/Menu.html
| - docs/components/NM2_Upgrade.html
| - docs/components/ZFS_Upgrade.html

INTERNAL

Pre-requisites

PCIe Renumbering check on ZS3-ES storage heads in Exalogic x4-2 and x5-2 racks

Refer to following Note which has information on this important pre-requisite check.

<Note 2087741.1>: Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks

Appendices

Appendix A: Fixed Bugs List

Please review Document ID : 2093677.1 - Exalogic Infrastructure January 2016 PSU - Fixed Bugs List.

Appendix B: Patching Known Issues

PrePatchCheck action, fails with "check /etc/inittab for ttyS0" , "check /etc/securetty for ttyS0"

Symptoms

prePatchCheck Fails with following message:

ERROR: Thu Dec 3 09:52:56 EST 2015: Expected ttyS0 entry missing in /etc/inittab
ERROR: Thu Dec 3 09:52:56 EST 2015: Expected ttyS0 entry missing in /etc/securetty

Cause

ttyS0 entry is not found

Solution/Workaround

1) Verify whether the /etc/inittab file has the following entry for ttyS0 above the line for tty1:

co:2345:respawn:/sbin/agetty ttyS0 9600 vt100-nav

If the entry does not exist, add it:

----- inittab ----------
...
# Run gettys in standard runlevels
co:2345:respawn:/sbin/agetty ttyS0 9600 vt100-nav
1:2345:respawn:/sbin/mingetty tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6
...

2) Verify whether the /etc/securetty file has an entry for ttyS0. If the entry does not exist, append it by running the following command:

[root@compute-node ~]# echo "ttyS0" >> /etc/securetty

3) Once you updated either /etc/inittab or /etc/securetty, reboot the compute nodes.

[root@compute-node ~]# reboot

ZFS patching fails due to losing IO access to the pool.

Symptoms

ZFS patching fails and Exapatch terminated as follows ...

INFO: ZFS-Storage-Head xx.xx.xx.xx successfully completed all pre-patch
checks
ERROR: ZFS-Storage-Head xx.xx.xx.xx STOP!! This head in pool exalogic is not
online.
Unable to proceed as pre-patch checks failed for active head xx.xx.xx.xx
Additional information may be found in the console output and the log file:
/var/log/exapatch_20151119112342.log

Observed in the exapatch log, errors are similar to the following:-

Head in Active state, continuing...
STOP!! This head in pool exalogic is not online.
STATUS : degraded
]]
[2015-11-19T11:53:27.441-05:00] [exapatch] [TRACE:1] [] [utils] [pid: 14494] [tid: MainThread] [ecid: ] [lineno: 219] [[
STDERR:
1
STOP!! This head in pool exalogic is not online.
STATUS : degraded
]]
[2015-11-19T11:53:27.441-05:00] [exapatch] [NOTIFICATION:1] [] [utils] [pid: 14494] [tid: MainThread] [ecid: ] [lineno: 340] ERROR: ZFS-Storage-Head xx.xxx.xx.xx STOP!! This head in pool exalogic is not online.
[2015-11-19T11:53:27.441-05:00] [exapatch] [NOTIFICATION:1] [] [utils] [pid: 14494] [tid: MainThread] [ecid: ] [lineno: 340] Unable to proceed as pre-patch checks failed for active head xx.xxx.xx.xx

Cause

During the patching, we found the system in a strange situation where the storage pool was owned by the passive(STRIPPED) head:

el01sn01:configuration storage> get

pool = exalogic

status = exported

owner = el01sn01

el01sn01:> configuration cluster show

Properties:

state = AKCS_OWNER << Active head doesn't own the pool

description = Active (takeover completed)

peer_asn = 2058f7ae-0247-c86e-c2f1-94f8cd68faa7

peer_hostname = el01sn02

peer_state = AKCS_STRIPPED

peer_description = Ready (waiting for failback)

Children:

resources => Configure resources

Node el01sn02 was READY (PASSIVE) but the storage was online on this.

el01sn02:configuration storage> get << this is the passive head

pool = exalogic

status = online

errors = 0

owner = el01sn01

profile = mirror

log_profile = log_stripe

cache_profile = cache_stripe

scrub = resilver completed after 0h0m with 0 errors

As the result, all clients are losing IO access to the pool.

Solution/Workaround

Reboot the active head and retry the patching

Refer <Note 1910623.1>

psuSetup.sh failing due to lack of free space.

Symptoms:

psusetup.sh fails due to lack of free space. It fails while reserving share quota to 200M for exalogic-lctools.

[root@compute-node]# ./psuSetup.sh <zfs_ip>

INFO: xx.xx.xx.xxx is a ZFS appliance
INFO: The IPoIB of the ZFS is 192.168.6.131
INFO: /exalogic-lctools does not appear to be mounted from xx.xx.xx.xxx

@ INFO: using existing "common" on the ZFS Appliance with IP address
xx.xx.xx.xxx
@ INFO: using existing "exalogic-lctools" on the ZFS Appliance with IP address
xx.xx.xx.xxx
ERROR: (failed to commit changes (encountered while attempting to run command "commit"))

Cause:

When it is trying to set the limit, it was already over the current available limit.

Solution/Workaround:

Free Up space In ZFSA. Refer <Note 2039102.1>

_____________________________________________________________________________________________________________________________________________________________________________

PCIE fault HW alert in ZFS post IB firmware update

Symptoms:

PCIE related HW fault is reported in ZFS, after applying IB card firmware.

Error message is Similar to:-

SUNW-MSG-ID: SPX86-8003-QH, TYPE: Fault, VER: 1, SEVERITY: Major
EVENT-TIME: Mon Aug 17 06:34:36 2015
PLATFORM: i86pc, CSN: 1441NML0FE, HOSTNAME: denp04sn02
SOURCE: appliance/kit/akd:default, REV: 1.0
EVENT-ID: d63362ca-89ea-e1e9-90cb-ee2bc152e335
DESC: An Integrated I/O (II0) non-fatal error in downstream PCIE device has
occurred.
AUTO-RESPONSE: The service-required LED on the chassis will be illuminated.
IMPACT: System continues to run with degraded resources.
REC-ACTION: Contact your service provider for proper repair procedures.

Cause:

A Hermon IB card firmware upgrade was active on the card.

Solution/Workaround:

Implement the workaround mentioned in <Note 2040757.1>

_____________________________________________________________________________________________________________________________________________________________________________________

Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Refer to <Note 2114516.1> for details on this known issue.

_____________________________________________________________________________________________________________________________________________________________________________________

Exalogic: ZFS-7320 ILOM Network Settings Lost on X2-2 and X3-2 Racks After PSU Upgrade

Refer to <Note 2115679.1> for details on this known issue.

_____________________________________________________________________________________________________________________________________________________________________________________

Exalogic X5-2 OL 6.5 Linux Physical Compute Node PSU Upgrade To 2.0.6.2.4 Jan 2016 PSU Failing With "FAILED - Running grub-install on /dev/sda" Error In Exapatch Logs

Refer to <Note 2115937.1> for details on this known issue.

Appendix: C

Exapatch with '-f' or '--force' option

Refer to below Note

<Note 1997466.1>: Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching

References

<NOTE:1314535.1> - Exalogic Patch Set Updates (PSU) Master Note
<NOTE:2092033.1> - Exalogic Compute Node or Guest vServer PSU Upgrade Using Exapatch Fails With Error "IndexError: list index out of range"
<NOTE:2114516.1> - Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade
<NOTE:1329262.1> - How to Perform a Healthcheck on Exalogic
<NOTE:2115937.1> - Exalogic X5-2 OL 6.5 Linux Physical Compute Node PSU Upgrade To 2.0.6.2.4 Jan 2016 PSU Failing With "FAILED - Running grub-install on /dev/sda" Error In Exapatch Logs
<NOTE:2115679.1> - Exalogic: ZFS-7320 ILOM Network Settings Lost on X2-2 and X3-2 Racks After PSU Upgrade
<NOTE:2040757.1> - Oracle ZFS Storage Appliance: How To Resolve - An Integrated I/O non-fatal error in downstream PCIE device has occurred.
<NOTE:1449226.1> - Exachk Health-Check Tool for Exalogic
<NOTE:2087741.1> - Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks

Attachments

This solution has no attachment