Exalogic Patch Set Update (PSU) Release 2.0.6.2.2 (Linux

Asset ID:	1-79-2027108.1
Update Date:	2016-04-21
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 2027108.1 : Exalogic Patch Set Update (PSU) Release 2.0.6.2.2 (Linux - Physical) for July 2015

Applies to:

Oracle Exalogic Elastic Cloud Software - Version 2.0.0.0.0 to 2.0.6.2.1
Oracle Exalogic Elastic Cloud X2-2 Hardware - Version X2 to X2 [Release X2]
Exalogic Elastic Cloud X3-2 Hardware
Exalogic Elastic Cloud X4-2 Hardware - Version X4 to X4 [Release X4]
Exalogic Elastic Cloud X5-2 Hardware
Linux x86-64

Purpose

Oracle Exalogic is an integrated hardware and software system designed to provide a complete platform for a wide range of application types and widely varied workloads. It combines optimized Oracle Fusion Middleware software like WebLogic server, JRockit, Coherence with industry-standard Sun Server and Storage hardware and InfiniBand networking. The purpose of this document is to provide specific information around July 2015 Patch Set Update (PSU) for that system.

Scope

The target audience of this document are engineers and system administrators who plan to apply the Exalogic PSU.

This document provides the following:

README files that are included in the July 2015 PSU for convenience. These can be used for reference without having to download the PSU first.
Known issues with the PSU - these provide a list of known issues along with workarounds/solutions, if available. Engineers and administrators planning to apply the PSU should ensure to review these before applying the PSU.

This document will be kept up-to-date with updates to errata and known issues.

Details

NOTE:

This PSU contains upgrades to both OL5, OL6 releases.
This PSU includes updates to ELLC 14.2.3

Patch Download

Released: July 2015

Product Version: 2.0.6.2.2 (on X2-2/X3-2/X4-2/X5-2) for Oracle Linux x86-64

PATCH 20860046 - EXALOGIC LINUX 2.0.6.2.2 (on X2-2/X3-2/X4-2/X5-2) PATCH SET UPDATE (PSU) FOR JULY 2015

Patch Readme Documentation

Refer to the readme documentation attached 20860046-Linux.zip to this Document on how to upgrade the Exalogic infrastructure:

Exalogic X2-2,X3-2,X4-2 and X5-2 Systems: Upgrading to 2.0.6.2.2

The readme content structure layout of the 20860046-Linux.zip:

20860046-Linux
|
| - README.html
| - exalogic-lctools-14.2.3-release-notes.txt
| - docs/README.html
| - docs/components/ILOM_ComputeNode_Upgrade.html
| - docs/components/Index.html
| - docs/components/Menu.html
| - docs/components/NM2_Upgrade.html
| - docs/components/ZFS_Upgrade.html

Pre-requisites

PCIe Renumbering check on ZS3-ES storage heads in Exalogic x4-2 and x5-2 racks

Refer to following Note which has information on this important pre-requisite check.

<Note 2087741.1>: Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks

Appendices

Appendix A: Fixed Bugs List

Please review Document ID : 2027076.1 - Exalogic Infrastructure July 2015 PSU - Fixed Bugs List.

Appendix B: Patching Known Issues

Compute node upgrade fails on using force option

Symptoms

Exapatch will fail at Installing additional packages. We can see from exapatch_xxxx.log, " ERROR: FAILED - Installing additional packages" from /var/log/init-exalogic-node/init-exalogic-node.log:

[EXALOGIC_ERROR:GENERAL] Jun 04 18:35:06 install.sh: Unsupported CX Card: type:
[EXALOGIC_ERROR:GENERAL] Jun 04 18:35:06 install.sh: Exalogic imaging failed
[EXALOGIC_ERROR:GENERAL] Jun 04 18:35:06 install.sh: supported_config file missing for this CX card type:

Cause

ibstat cmd didn't show any output at that instant point of time. However if we manually run it, it shows up fine.

Solution/Workaround

Reboot and Re-run the exapatch cmd once again to patch successfully.

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

CN UPGRADE FAILED : ipmitool sunoem version reports error

Symptoms

ipmitool commands fail with errors similar to the following:

[root@compute-node ~]# ipmitool sunoem version
Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory

Inspecting the /var/log/exapatch_xxxx.log:

Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory
Sun OEM Get SP Version Failed.

Cause

If the ILOM is hung and unresponsive, it's IPMI device interface may no longer be accessible.

Solution/Workaround

Restart the ipmi services by doing following, and retry the patching

[root@compute-node ~]# service ipmi restart
Stopping all ipmi drivers: [ OK ]
Starting ipmi drivers: [ OK ]

[root@compute-node ~]# service ipmi status
ipmi_msghandler module loaded.
ipmi_si module loaded.
ipmi_devintf module loaded.
/dev/ipmi0 exists.

Note: If the error still persist after multiple attempts of above procedure, the ILOM/compute node will have to reimaged

ZFS Upgrade Failed. Storage node ilom console indicates error in joining cluster.

Symptoms

While applying the July 2015 PSU on X5-2 rack, the upgrade of ZS3-ES may fail. The exapatch fails with:-

"ERROR: Could not ssh to ZFS host xx.xx.xx.xx after upgrade. "

Further,

When I see the storage node ilom of the failed head, it is stuck at this stage :

Oracle ZFS Storage ZS3-ES Configuration
    Copyright (c) 2008, 2015, Oracle and/or its affiliates. All rights
reserved.

    NET-0 <=>  NET-1 <=>  NET-2 <=>  NET-3 <X>

    SUNW-MSG-ID: AK-8000-2U, TYPE: Defect, VER: 1, SEVERITY: Major
    EVENT-TIME: Tue Jun  2 17:01:26 UTC 2015
    PLATFORM: i86pc, CSN: 1427NML0E5, HOSTNAME: scat01sn02
    SOURCE: aktty, REV: 1.0
    EVENT-ID: 1fc05eee-db80-e94d-d6c2-c06714389111
    DESC: The appliance experienced an unrecoverable protocol error while
    attempting to join a cluster of appliances.  Refer to
    http://sun.com/msg/AK-8000-2U for more information.
    AUTO-RESPONSE: No automated response is possible for this defect.
    IMPACT: The appliance cannot be configured into the cluster.
    REC-ACTION: Reboot the system and attempt to configure the cluster again.
    Contact your vendor for support.

    ESC-3: Halt   ESC-4: Reboot   ESC-5: Info

    For help, see http://www.oracle.com/goto/ZS3-ES/docs/

Cause

@ Details of this issue are in this MOS note:
@ .
@ Oracle ZFS Storage Appliance: ZS3-X Clustron Card in PCIE Slot 2 Does Not
@ Initialize During boot. (Doc ID 1910623.1)
@ .
@ The following bug is referenced from the MOS note:
@ .
@ BUG 18672251 - AK-8000-2U: unrecoverable protocol error on the ZS3-ES
@ (X4170M3) appliance

Solution/Workaround

Manually reboot the storage head with ESC-4. Refer to <Note 1910623.1>.

PSU Upgrade fails while upgrading ZFSSA.

Symptoms: While upgrading ZFS software from 2011.x to 2013.x using exapatch, the upgrade fails after upgrading one of the two storage control heads.

The following error is observed in the exapatch logs:

COMPONENT DIAGNOSED TYPE DESCRIPTION
problem-000 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS1025TM6Z/a'
in pool 'exalogic' failed.

problem-001 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS1035TM6Z/a'
in pool 'exalogic' failed.

problem-002 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS102FTM6Z/a'
in pool 'exalogic' failed.

problem-003 2015-5-1 17:45:07 Major Fault ZFS device

Pre-patch check fails on the second storage head causing exapatch to exit the upgrade operation.

Cause: The pre-patch check failure is due to a known issue tracked through the following ZFS bug:

BUG 19024367 - Peer node's RZ listed under problems and new problem entry is created on repair

- Readzilla belonging to peer node are marked faulty and listed under
problems
- When an attempt is made to repair the unavailable readzilla entry, one more problem
entry gets added

A fix for the above bug is not yet available for Exalogic. Implement the following workaround to proceed with the ZFS upgrade.

Solution/Workaround:

Since Exapatch does not allow you to proceed, the software on the of the remaining head must be upgraded manually.

The following Exalogic patch contains a manual procedure for the ZFS software upgrade

https://updates.oracle.com/download/20983458.html

___________________________________________________________________________________________________________________________________________________________________

PSU Fails to Patch Compute Nodes Due to a Conflict with a file from package vmpinfo-sosreport-1.0.3-1.noarch

Refer to <Note 2078002.1> for details.

______________________________________________________________________________________________________________________________________________________________________

Exalogic Compute Node or Guest vServer PSU Upgrade Using Exapatch Fails With Error "IndexError: list index out of range"

Refer to <Note 2092033.1> for details on this known issue.

_____________________________________________________________________________________________________________________________________________________________________________________

Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Refer to <Note 2114516.1> for details on this known issue.

___________________________________________________________________________________________________________________________________________________________________

Appendix C: FAQs

Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching

Refer to <Note 1997466.1> for details

References

<NOTE:1329262.1> - How to Perform a Healthcheck on Exalogic
<NOTE:2078002.1> - PSU Fails to Patch Compute Nodes Due to a Conflict with a file from package vmpinfo-sosreport-1.0.3-1.noarch
<NOTE:1449226.1> - Exachk Health-Check Tool for Exalogic
<NOTE:1314535.1> - Exalogic Patch Set Updates (PSU) Master Note
<NOTE:2068546.1> - On Exalogic Physical Compute Nodes Interface Hardware Addresses Are Set To Zeros (00:00) For IPoIB Interfaces Using Custom Partitions After Applying Either April Or July 2015 PSU
<NOTE:2087741.1> - Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks
<NOTE:2092033.1> - Exalogic Compute Node or Guest vServer PSU Upgrade Using Exapatch Fails With Error "IndexError: list index out of range"
<NOTE:2114516.1> - Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Attachments

This solution has no attachment