Exalogic Patch Set Update (PSU) Release 2.0.6.2.2 (Linux

Asset ID:	1-79-2027098.1
Update Date:	2018-01-22
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 2027098.1 : Exalogic Patch Set Update (PSU) Release 2.0.6.2.2 (Linux - Virtual) for July 2015

Applies to:

Exalogic Elastic Cloud X3-2 Hardware
Exalogic Elastic Cloud X5-2 Hardware
Oracle Exalogic Elastic Cloud Software - Version 2.0.6.0.0 to 2.0.6.2.1
Oracle Exalogic Elastic Cloud X2-2 Hardware
Exalogic Elastic Cloud X4-2 Hardware
Linux x86-64
Oracle Virtual Server(x86-64)

Purpose

Oracle Exalogic is an integrated hardware and software system designed to provide a complete platform for a wide range of application types and widely varied workloads. It combines optimized Oracle Fusion Middleware software like WebLogic server, JRockit, Coherence with industry-standard Sun Server and Storage hardware and InfiniBand networking. The purpose of this document is to provide specific information around July 2015 Patch Set Update (PSU) for that system.

Scope

The target audience of this document are engineers and system administrators who plan to apply the Exalogic PSU.

This document provides the following:

README files that are included in the July 2015 PSU for convenience. These can be used for reference without having to download the PSU first
Known issues with the PSU - these provide a list of known issues along with workarounds/solutions, if available. Engineers and administrators planning to apply the PSU should ensure to review these before applying the PSU

This document will be kept up-to-date with updates to errata and known issues.

Details

Note:

This PSU it includes updates to ELLC 14.2.3
This PSU includes updates for vservers based on OL5 and OL6 templates

Patch Download

Released: July 2015

Product Version: 2.0.6.2.2 (on X2-2/X3-2/X4-2/X5-2) for Oracle Exalogic Elastic Cloud infrastructure

PATCH 20859979 - EXALOGIC VIRTUAL 2.0.6.2.2 (on X2-2/X3-2/X4-2/X5-2) PATCH SET UPDATE (PSU) FOR JULY 2015

Patch Readme Documentation

Refer to the readme documentation attached 20859979-Virtual.zip to this Document on how to upgrade the Exalogic infrastructure:

Exalogic X2-2, X3-2,X4-2 and X5-2 Systems: Upgrading to 2.0.6.2.2

The readme content structure layout of the 20859979-Virtual.zip:

20859979-Virtual
|
| - README.html
| - exalogic-lctools-14.2.3-release-notes.txt
| - docs/README.html
| - docs/components/ECServices_Upgrade.html
| - docs/components/ILOM_ComputeNode_Upgrade.html
| - docs/components/Index.html
| - docs/components/Menu.html
| - docs/components/NM2_Upgrade.html
| - docs/components/VServers_Upgrade.html
| - docs/components/ZFS_Upgrade.html

Pre-requisites

O2CB Cluster Timeout

Before patching, verify that the cluster heartbeat timeout is set correctly, as documented in the Document ID : 1995593.1 - Increase O2CB Cluster Heartbeat Timeout on Exalogic Virtual

PCIe Renumbering check on ZS3-ES storage heads in Exalogic x4-2 and x5-2 racks

Refer to following Note which has information on this important pre-requisite check.

<Note 2087741.1>: Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks

Appendices

Appendix A: Fixed Bugs List

Please review Doc ID 2027076.1: Exalogic Infrastructure July 2015 PSU – Fixed Bugs List

Appendix B: Patching Known Issues

The pre-requisite step to update Weblogic Server configuration properties for non-default passwords prior to upgrading Exalogic Control Services fails

Symptoms

A pre-requisite step is included in section entitled "1.1 Prerequisites" to configure Weblogic Server prior to upgrading Exalogic Control Services to handle non-default passwords configured for the weblogic user. Running the command as documented results in an error as shown below:

[oracle@elctrl ~]$ /u01/app/oracle/ovm-manager-3/weblogic/wlconfig.sh weblogic <non-default-password>

-bash: /u01/app/oracle/ovm-manager-3/weblogic/wlconfig.sh: Permission denied

Invoking it using sh as follows runs into ClassNotFoundException, as shown in the sample below:

[oracle@elctrl ~]$ sh /u01/app/oracle/ovm-manager-3/weblogic/wlconfig.sh weblogic welcome1
Exception in thread "main" java.lang.NoClassDefFoundError: weblogic.WLST
   at gnu.java.lang.MainThread.run(libgcj.so.7rh)
Caused by: java.lang.ClassNotFoundException: weblogic.WLST not found in gnu.gcj.runtime.SystemClassLoader{urls=[file:./], parent=gnu.gcj.runtime.ExtensionClassLoader{urls=[], parent=null}}
   at java.net.URLClassLoader.findClass(libgcj.so.7rh)
   at java.lang.ClassLoader.loadClass(libgcj.so.7rh)
   at java.lang.ClassLoader.loadClass(libgcj.so.7rh)
   at gnu.java.lang.MainThread.run(libgcj.so.7rh)
[oracle@elctrl ~]

Cause

A step to set the WLS environment variables is missing in the documentation. Also, the wlconfig.sh script is not executable; it needs to be invoked using "sh".

Solution/Workaround

Login as root to the Exalogic Control VServer
Switch to user oracle

[root@elctrl ~]# su - oracle
[oracle@elctrl ~]$
Run the following command to set the WLS environment variables:

[oracle@elctrl ~]$ cd /u01/app/oracle/Middleware/wlserver_10.3/server/bin
[oracle@elctrl bin]$ . ./setWLSEnv.sh

CLASSPATH=/u01/app/oracle/Middleware/patch_wls1035/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/u01/app/oracle/Middleware/patch_ocp360/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/u01/app/oracle/java/lib/tools.jar:/u01/app/oracle/Middleware/wlserver_10.3/server/lib/weblogic_sp.jar:/u01/app/oracle/Middleware/wlserver_10.3/server/lib/weblogic.jar:/u01/app/oracle/Middleware/modules/features/weblogic.server.modules_10.3.5.0.jar:/u01/app/oracle/Middleware/wlserver_10.3/server/lib/webservices.jar:/u01/app/oracle/Middleware/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/u01/app/oracle/Middleware/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:

PATH=/u01/app/oracle/Middleware/wlserver_10.3/server/bin:/u01/app/oracle/Middleware/modules/org.apache.ant_1.7.1/bin:/u01/app/oracle/java/jre/bin:/u01/app/oracle/java/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/oracle/bin

Your environment has been set.

NOTE: Ensure that the above command is run with a dot and a space preceding ./setELSEnv.sh as shown.
Run the wlsconfig.sh command to configure the non-default password for the weblogic user

[oracle@elctrl bin]$ cd /u01/app/oracle/ovm-manager-3/weblogic
[oracle@elctrl weblogic]$ sh ./wlconfig.sh weblogic <non-default-password>

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

Connecting to t3://localhost:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'base_adf_domain'.

Warning: An insecure protocol was used to connect to the
server. To ensure on-the-wire security, the SSL port or
Admin port should be used instead.

Using existing user key file...
The username and password that were used for this WebLogic Server connection are stored in wlconfig.properties and wlkey.properties.
[oracle@elctrl weblogic]$

After above steps you may proceed with updating Exalogic Control Services.

Exapasswd: EMOC sync fails for Exapasswd with special characters.

Symptoms

If exapasswd is used with certain special characters synchronization with EMOC would fail. Observed following error messages:

Error while synchronizing passwords with Ops Center

Cause

The issue is with following special characters:

1) ` (acute or back quote)
2) : (colon)
3) ; (semi-colon)

Solution/Workaround

It is known limitation that exapasswd does not support quotes (double/single) also. And it has already been documented.

These new findings will also be updated in the Expasswd user's guide accordingly.

ZFS Upgrade Failed. Storage node ilom console indicates error in joining cluster.

Symptoms

While applying the July 2015 PSU on X5-2 rack, the upgrade of ZS3-ES may fail. The exapatch fails with:-

"ERROR: Could not ssh to ZFS host xx.xx.xx.xx after upgrade. "

Further,

When I see the storage node ilom of the failed head, it is stuck at this stage :

Oracle ZFS Storage ZS3-ES Configuration
    Copyright (c) 2008, 2015, Oracle and/or its affiliates. All rights
reserved.

    NET-0 <=>  NET-1 <=>  NET-2 <=>  NET-3 <X>

    SUNW-MSG-ID: AK-8000-2U, TYPE: Defect, VER: 1, SEVERITY: Major
    EVENT-TIME: Tue Jun  2 17:01:26 UTC 2015
    PLATFORM: i86pc, CSN: 1427NML0E5, HOSTNAME: scat01sn02
    SOURCE: aktty, REV: 1.0
    EVENT-ID: 1fc05eee-db80-e94d-d6c2-c06714389111
    DESC: The appliance experienced an unrecoverable protocol error while
    attempting to join a cluster of appliances.  Refer to
    http://sun.com/msg/AK-8000-2U for more information.
    AUTO-RESPONSE: No automated response is possible for this defect.
    IMPACT: The appliance cannot be configured into the cluster.
    REC-ACTION: Reboot the system and attempt to configure the cluster again.
    Contact your vendor for support.

    ESC-3: Halt   ESC-4: Reboot   ESC-5: Info

    For help, see http://www.oracle.com/goto/ZS3-ES/docs/

Cause

@ Details of this issue are in this MOS note:
@ .
@ Oracle ZFS Storage Appliance: ZS3-X Clustron Card in PCIE Slot 2 Does Not
@ Initialize During boot. (Doc ID 1910623.1)
@ .
@ The following bug is referenced from the MOS note:
@ .
@ BUG 18672251 - AK-8000-2U: unrecoverable protocol error on the ZS3-ES
@ (X4170M3) appliance

Solution/Workaround

Manually reboot the storage head with ESC-4. Refer to <Note 1910623.1>

PSU Upgrade fails while upgrading ZFSSA.

Symptoms: While upgrading ZFS software from 2011.x to 2013.x using exapatch, the upgrade fails after upgrading one of the two storage control heads.

The following error is observed in the exapatch logs:

COMPONENT DIAGNOSED TYPE DESCRIPTION
problem-000 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS1025TM6Z/a'
in pool 'exalogic' failed.

problem-001 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS1035TM6Z/a'
in pool 'exalogic' failed.

problem-002 2015-5-1 17:45:07 Major Fault ZFS device
'id1,sd@SATA_____TOSHIBA_THNSNC51________52VS102FTM6Z/a'
in pool 'exalogic' failed.

problem-003 2015-5-1 17:45:07 Major Fault ZFS device

Pre-patch check fails on the second storage head causing exapatch to exit the upgrade operation.

Cause: The pre-patch check failure is due to a known issue tracked through the following ZFS bug:

BUG 19024367 - Peer node's RZ listed under problems and new problem entry is created on repair

- Readzilla belonging to peer node are marked faulty and listed under
problems
- When an attempt is made to repair the unavailable readzilla entry, one more problem
entry gets added

A fix for the above bug is not yet available for Exalogic. Implement the following workaround to proceed with the ZFS upgrade.

Solution/Workaround:

Since Exapatch does not allow you to proceed, the software on the of the remaining head must be upgraded manually.

The following Exalogic patch contains a manual procedure for the ZFS software upgrade

https://updates.oracle.com/download/20983458.html

xm list fails to run on compute nodes upgraded from April 2015 PSU (2.0.6.2.1) to July 2015 PSU (2.0.6.2.2) causing ovm refresh failure

Symptoms

After patching a compute node from April 2015 PSU (2.0.6.2.1) to July 2015 PSU (2.0.6.2.2), refresh of server pools in OVMM console fails with the below error.

Job Construction Phase
----------------------
Job ID: 1437389471836

begin()
Appended operation 'Server Refresh' to object
'ff:20:00:08:ff:ff:ff:ff:ff:ff:00:21:28:af:ec:2c
(esct-tvp540-cn07.uk.oracle.com)'.
commit()
Completed Step: COMMIT

Objects and Operations
----------------------
Object (IN_USE): [Server] ff:20:00:08:ff:ff:ff:ff:ff:ff:00:21:28:af:ec:2c
(esct-tvp540-cn07.uk.oracle.com)
Operation: Server Refresh
.
Job Running Phase at 2015-07-20 11:51:11,836
----------------------------------------------
Job Participants: []
.
.
Actioner
--------
11:51:12,062: Starting operation 'Server Refresh' on object
'ff:20:00:08:ff:ff:ff:ff:ff:ff:00:21:28:af:ec:2c
(esct-tvp540-cn07.uk.oracle.com)'
Setting Context to model only in job with id=1437389471836
Job Internal Error
(Operation)com.oracle.ovm.mgr.api.exception.FailedOperationException:
OVMAPI_4010E Attempt to send command: dispatch to server:
esct-tvp540-cn07.uk.oracle.com failed. OVMAPI_4004E Server Failed Command:
dispatch https://?uname?:?pwd?@192.168.23.7:8899/api/3 list_vms, Status:
org.apache.xmlrpc.XmlRpcException: exceptions.RuntimeError:Command: ['xm',
'list', '--long'] failed (1): stderr: Error: domains_with_state() takes
exactly 3 arguments (4 given)
stdout: Usage: xm list [options] [Domain, ...]
.
List information about all/some domains.
-l, --long Output all VM details in SXP
--label Include security labels
--state= Select only VMs with the specified state
--pool= Select only VMs in specified cpu pool

Cause

A node reboot is required following the update of xen RPMs performed as part of the PSU, but the patching script does not perform a reboot for upgrade from April 2015 PSU to July 2015 PSU. Without the node reboot, xm list command fails on the upgraded node, causing the OVM refresh failure.

Solution/Workaround

Apply the following Patch 21477745 to the July 2015 PSU bits prior to upgrading nodes at April 2015 PSU to July 2015 PSU to avoid the problem:

BUG 21477745 - INTERIM PATCH WITH FIX FOR OVM SERVER POOL REFRESH FAILURE IN JULY 2015 PSU.

Note: The patch updates the PSU patching scripts, so it is important for the July 2015 PSU to be downloaded and extracted prior to applying this patch.

Note: This patch is not applicable to patching flows other than April 2015 PSU to July 2015 PSU.

If the patch is not applied prior to upgrading to July 2015 PSU, it may also be used to recover from the problematic state.

Guest vServer update fails during yum update of rpms with 'up2dateErrors'

Refer to Exalogic PSU Known Issues <Note 1571367.1>, section "Guest vServer update fails during yum update of rpms with 'up2dateErrors'" for information on this known issue.

Receiving Error When Patching Guest vServer with Exalogic PSU - ERROR: Missing "--uri URI-to-patch" option

Refer to <Note 2078653.1> for details.

___________________________________________________________________________________________________________________________________________________

PSU Fails to Patch Compute Nodes Due to a Conflict with a file from package vmpinfo-sosreport-1.0.3-1.noarch

Refer to <Note 2078002.1> for details.

___________________________________________________________________________________________________________________________________________________

Receiving Error When Patching Guest vServer with Exalogic PSU - ERROR: Missing "--uri URI-to-patch" option

Refer to <Note 2078653.1> for details.

______________________________________________________________________________________________________________________________________________________________________

Exalogic Compute Node or Guest vServer PSU Upgrade Using Exapatch Fails With Error "IndexError: list index out of range"

Refer to <Note 2092033.1> for details on this known issue.

_____________________________________________________________________________________________________________________________________________________________________________________

Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Refer to <Note 2114516.1> for details on this known issue.

Appendix C: Errata

1. Migration of assets managed by Proxy Controllers

In the troubleshooting MOS note section "Problem: components are listed under both the ProxyControllers(PC)"

Original Text

In the Exalogic Control BUI, if it is observed that any of the component asset is listed under both the Proxy Controllers (PC), they need to be migrated to a single PC.

Updated Text

For a given switch (NM2-GW switches, NM2-36p switch), only one proxy controller needs to be managing it. If a particular switch appears to be managed by both PC1 and PC2, it must be migrated to one of the proxy controllers. It does not matter whether it is migrated to PC1 or PC2.

One or more compute nodes may appear in the Managed Assets list of both proxy controllers. For instance, el01cn01.example.com may appear as being managed by both PC1 and PC2. This is expected behavior; no migration is required for the compute nodes.

Other non-switch assets, such as ZFS storage heads and PDU may also appear in the Managed Assets list of both proxy controllers. Migration is not required for these assets prior to applying the July 2015 PSU; migration is required only for switches.

Login to EMOC BUI and navigate to "Administration" item in the left panel and find the entries for 'PC1' and 'PC2' vServers.

Select a Proxy Controller say, "PC1" in the left panel. In the center panel click on the "Managed Assets" tab and set the Asset Type Filter to "Network Switches" to get list of switches managed by the 'PC1' Proxy Controller.

Select the switch that you wish to migrate to the other ProxyController "PC2". Click on the icon that provides the option to "Migrate Assets". A confirmation dialog shows up, select 'Migrate' button to proceed.

Once the migration completes finishes, a notification pop-up appears at the bottom right corner of the EMOC BUI, confirming the successful migration.

2. Exapatch console output for ZFS upgrade

Original Console Output Text

# /exalogic-lctools/bin/exapatch -a patch zfs_software
Logging to file /var/log/exapatch_20141107021834.log
INFO: ZFS-Storage-Head xx.xx.xx.1 running pre-patch checks for active head
INFO: ZFS-Storage-Head xx.xx.xx.1 Active Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all pre-patch checks
INFO: ZFS: Rebooting xx.xx.xx.1
INFO: ZFS: xx.xx.xx.1 Head, reboot command issued
INFO: ZFS: Waiting for head xx.xx.xx.1 to restart after reboot
INFO: ZFS: Running on xx.xx.xx.2 , Waiting for other head after reboot
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_OWNER, Peer Description - Active (takeover completed)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: Peer head is ready with State - AKCS_STRIPPED, Description - Ready (waiting for failback)
INFO: ZFS: Host xx.xx.xx.1 is now back online, waiting for sshd to start
INFO: ZFS: Host xx.xx.xx.1 is now accessible via ssh
INFO: ZFS-Storage-Head xx.xx.xx.2 running pre-patch checks for active head
INFO: ZFS-Storage-Head xx.xx.xx.2 Active Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all pre-patch checks
INFO: ZFS: Rebooting xx.xx.xx.2
INFO: ZFS: xx.xx.xx.2 Head, reboot command issued
INFO: ZFS: Waiting for head xx.xx.xx.2 to restart after reboot
INFO: ZFS: Running on xx.xx.xx.1 , Waiting for other head after reboot
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_OWNER, Peer Description - Active (takeover completed)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: Peer head is ready with State - AKCS_STRIPPED, Description - Ready (waiting for failback)
INFO: ZFS: Host xx.xx.xx.2 is now back online, waiting for sshd to start
INFO: ZFS: Host xx.xx.xx.2 is now accessible via ssh
INFO: ZFS-Storage-Head xx.xx.xx.2 running pre-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.2 Ready Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all pre-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 Active Head PreReq Checks Completed.
Upgrading software on ZFS-Storage-Head host xx.xx.xx.2 from version:
Appliance Version: 2013.06.05.2.9,1-1.1
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-006 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-007 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0INFO: ZFS: Download script instantiated, Now executing on Ready Head xx.xx.xx.2
INFO: ZFS: Running upgrade, WARNING: This command will reboot the current head at the end of the process
INFO: ZFS: Waiting for ZFS Head xx.xx.xx.2 to restart after reboot
INFO: ZFS: Host xx.xx.xx.2 is now accessible via SSH
Completed upgrade of software on ZFS-Storage-Head host xx.xx.xx.2 new version:
Appliance Version: 2013.06.05.3.5,1-1.1
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-006 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-007 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0INFO: ZFS: Host xx.xx.xx.2 patched successfully, proceeding with setting various parameters
INFO: ZFS: xx.xx.xx.2 mapid_domain set successfully
INFO: ZFS-Storage-Head xx.xx.xx.2 Configuring IPMP Service Parameters
INFO: ZFS: Configured IPMP Service Parameters.
INFO: ZFS-Storage-Head xx.xx.xx.2 Disabling NFSv4 delegation
INFO: ZFS: Disabled NFSv4 delegation.
INFO: ZFS-Storage-Head xx.xx.xx.2 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 running post-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.2 Ready Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all post-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 Active Head PreReq Checks Completed.
INFO: ZFS: Rebooting xx.xx.xx.1
INFO: ZFS: xx.xx.xx.1 Head, reboot command issued
INFO: ZFS: Waiting for head xx.xx.xx.1 to restart after reboot
INFO: ZFS: Running on xx.xx.xx.2 , Waiting for other head after reboot
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_OWNER, Peer Description - Active (takeover completed)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: Peer head is ready with State - AKCS_STRIPPED, Description - Ready (waiting for failback)
INFO: ZFS: Host xx.xx.xx.1 is now back online, waiting for sshd to start
INFO: ZFS: Host xx.xx.xx.1 is now accessible via ssh
INFO: ZFS-Storage-Head xx.xx.xx.1 running pre-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.1 Ready Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all pre-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 Active Head PreReq Checks Completed.
Upgrading software on ZFS-Storage-Head host xx.xx.xx.1 from version:
Appliance Version: 2013.06.05.2.9,1-1.1
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-006 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-007 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0
INFO: ZFS: Download script instantiated, Now executing on Ready Head xx.xx.xx.1
INFO: ZFS: Running upgrade, WARNING: This command will reboot the current head at the end of the process
INFO: ZFS: Waiting for ZFS Head xx.xx.xx.1 to restart after reboot
INFO: ZFS: Host xx.xx.xx.1 is now accessible via SSH
Completed upgrade of software on ZFS-Storage-Head host xx.xx.xx.1 new version:
Appliance Version: 2013.06.05.3.5,1-1.1
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-006 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-007 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0
INFO: ZFS: Host xx.xx.xx.1 patched successfully, proceeding with setting various parameters
INFO: ZFS: xx.xx.xx.1 mapid_domain set successfully
INFO: ZFS-Storage-Head xx.xx.xx.1 Configuring IPMP Service Parameters
INFO: ZFS: Configured IPMP Service Parameters.
INFO: ZFS-Storage-Head xx.xx.xx.1 Disabling NFSv4 delegation
INFO: ZFS: Disabled NFSv4 delegation.
INFO: ZFS-Storage-Head xx.xx.xx.1 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 running post-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.1 Ready Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all post-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 Active Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 running post-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.1 Ready Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all post-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 running post-patch checks for active head
INFO: ZFS-Storage-Head xx.xx.xx.2 Active Head PreReq Checks Completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all post-patch checks
INFO: ZFS: Patching completed on both heads

Correct Console Output Text

# /exalogic-lctools/bin/exapatch -a patch zfs_software
Logging to file /var/log/exapatch_20150518183711.log
INFO: ZFS-Storage-Head xx.xx.xx.1 running pre-patch checks for active head
INFO: ZFS-Storage-Head xx.xx.xx.1 Active head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all pre-patch checks
INFO: ZFS: Rebooting xx.xx.xx.1
INFO: ZFS: xx.xx.xx.1 Head, reboot command issued
INFO: ZFS: Waiting for head xx.xx.xx.1 to restart after reboot
INFO: ZFS: Running on xx.xx.xx.2, waiting for other head to restart after reboot
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_OWNER, Peer Description - Active (takeover completed)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_BOOT_RX_JOIN, Peer Description - Rejoining cluster ...
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: Peer head is ready with State - AKCS_STRIPPED, Description - Ready (waiting for failback)
INFO: ZFS: Host xx.xx.xx.1 is now back online, waiting for sshd to start
INFO: ZFS: Host xx.xx.xx.1 is now accessible via ssh
INFO: ZFS-Storage-Head xx.xx.xx.2 running pre-patch checks for active head
INFO: ZFS-Storage-Head xx.xx.xx.2 Active head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all pre-patch checks
INFO: ZFS: Rebooting xx.xx.xx.2
INFO: ZFS: xx.xx.xx.2 Head, reboot command issued
INFO: ZFS: Waiting for head xx.xx.xx.2 to restart after reboot
INFO: ZFS: Running on xx.xx.xx.1, waiting for other head to restart after reboot
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_OWNER, Peer Description - Active (takeover completed)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_BOOT_RX_JOIN, Peer Description - Rejoining cluster ...
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: Peer head is ready with State - AKCS_STRIPPED, Description - Ready (waiting for failback)
INFO: ZFS: Host xx.xx.xx.2 is now back online, waiting for sshd to start
INFO: ZFS: Host xx.xx.xx.2 is now accessible via ssh
INFO: ZFS-Storage-Head xx.xx.xx.1 is active ZFS head.
INFO: ZFS-Storage-Head xx.xx.xx.2 is ready ZFS head.
INFO: ZFS-Storage-Head xx.xx.xx.2 running pre-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.2 Ready head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all pre-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 Active head checks completed.
Upgrading software on ZFS-Storage-Head host xx.xx.xx.2 from version:
Appliance Version: 2013.06.05.3.5,1-1.1
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-006 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-007 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0
INFO: ZFS: Download script instantiated, Now executing on Ready Head xx.xx.xx.2
INFO: ZFS: Running upgrade, WARNING: This command will reboot the current head at the end of the process
Completed upgrade of software on ZFS-Storage-Head host xx.xx.xx.2 new version:
Appliance Version: 2013.06.05.3.5,1-2.1.2.2
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-006 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-007 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0
INFO: ZFS: Host xx.xx.xx.2 patched successfully, proceeding with setting various parameters
INFO: ZFS: xx.xx.xx.2 mapid_domain set successfully
INFO: ZFS-Storage-Head xx.xx.xx.2 Configuring IPMP Service Parameters
INFO: ZFS: Configured IPMP Service Parameters.
INFO: ZFS-Storage-Head xx.xx.xx.2 Disabling NFSv4 delegation
INFO: ZFS: Disabled NFSv4 delegation.
INFO: ZFS-Storage-Head xx.xx.xx.2 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 running post-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.2 Ready head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all post-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 Active head checks completed.
ZFS-Storage-Head xx.xx.xx.2 Installing and/or executing ZFS workflow "Set NFS:server_enable_idmap to on/off"
ZFS-Storage-Head xx.xx.xx.2 Installed and/or executed ZFS workflow "Set NFS:server_enable_idmap to on/off"
INFO: ZFS: Rebooting xx.xx.xx.1
INFO: ZFS: xx.xx.xx.1 Head, reboot command issued
INFO: ZFS: Waiting for head xx.xx.xx.1 to restart after reboot
INFO: ZFS: Running on xx.xx.xx.2, waiting for other head to restart after reboot
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - AKCS_OWNER, Peer Description - Active (takeover completed)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: STOP! as the other head is not ready yet, Peer State - , Peer Description - Unknown (disconnected or restarting)
INFO: ZFS: Sleeping for 60 seconds
INFO: ZFS: Peer head is ready with State - AKCS_STRIPPED, Description - Ready (waiting for failback)
INFO: ZFS: Host xx.xx.xx.1 is now back online, waiting for sshd to start
INFO: ZFS: Host xx.xx.xx.1 is now accessible via ssh
INFO: ZFS-Storage-Head xx.xx.xx.2 is active ZFS head.
INFO: ZFS-Storage-Head xx.xx.xx.1 is ready ZFS head.
INFO: ZFS-Storage-Head xx.xx.xx.1 running pre-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.1 Ready head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all pre-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 Active head checks completed.
Upgrading software on ZFS-Storage-Head host xx.xx.xx.1 from version:
Appliance Version: 2013.06.05.3.5,1-1.1
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-006 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-007 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0
INFO: ZFS: Download script instantiated, Now executing on Ready Head xx.xx.xx.1
INFO: ZFS: Running upgrade, WARNING: This command will reboot the current head at the end of the process
Completed upgrade of software on ZFS-Storage-Head host xx.xx.xx.1 new version:
Appliance Version: 2013.06.05.3.5,1-2.1.2.2
BIOS: American Megatrends Inc. 08120104 05/08/2012
Service Processor: 3.0.16.10
workflow-000 Clear locks root false Oracle Corporation 1.0.0
workflow-001 Configure for Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-002 Unconfigure Oracle Solaris Cluster NFS root false Oracle Corporation 1.0.0
workflow-003 Increase to IB acache entries to 1024 root false Oracle Inc. undefined
workflow-004 Provide work around for CR 7122961 root false Oracle Inc. undefined
workflow-005 Set NFS:server_enable_idmap to on/off root false Oracle Inc. 1.0.0
workflow-006 Setting zil_replay_thread to one root false Oracle Inc. 1.1
workflow-007 Configure for Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.1
workflow-008 Unconfigure Oracle Enterprise Manager Monitoring root false Sun Microsystems, Inc. 1.0
INFO: ZFS: Host xx.xx.xx.1 patched successfully, proceeding with setting various parameters
INFO: ZFS: xx.xx.xx.1 mapid_domain set successfully
INFO: ZFS-Storage-Head xx.xx.xx.1 Configuring IPMP Service Parameters
INFO: ZFS: Configured IPMP Service Parameters.
INFO: ZFS-Storage-Head xx.xx.xx.1 Disabling NFSv4 delegation
INFO: ZFS: Disabled NFSv4 delegation.
INFO: ZFS-Storage-Head xx.xx.xx.1 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 running post-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.1 Ready head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all post-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 Active head checks completed.
ZFS-Storage-Head xx.xx.xx.1 Installing and/or executing ZFS workflow "Set NFS:server_enable_idmap to on/off"
ZFS-Storage-Head xx.xx.xx.1 Installed and/or executed ZFS workflow "Set NFS:server_enable_idmap to on/off"
INFO: ZFS-Storage-Head xx.xx.xx.2 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.2 running post-patch checks for active head
INFO: ZFS-Storage-Head xx.xx.xx.2 Active head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.2 successfully completed all post-patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 running post patch checks
INFO: ZFS-Storage-Head xx.xx.xx.1 running post-patch checks for ready head
INFO: ZFS-Storage-Head xx.xx.xx.1 Ready head checks completed.
INFO: ZFS-Storage-Head xx.xx.xx.1 successfully completed all post-patch checks

___________________________________________________________________________________________________________________

Appendix D: FAQs

Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching

Refer to <Note 1997466.1> for details

Guest vServer patching FAQ's

Refer to <Note 2031749.1> for details.

References

<NOTE:1449226.1> - Exachk Health-Check Tool for Exalogic
<NOTE:2016700.1> - Exalogic: vServer Creation Takes a Long Time or Fails After a Few Days of Uptime
<NOTE:2087741.1> - Patching ZS3-ES Renumbering Check On Exalogic x4-2 And x5-2 Racks
<NOTE:2078002.1> - PSU Fails to Patch Compute Nodes Due to a Conflict with a file from package vmpinfo-sosreport-1.0.3-1.noarch
<NOTE:1329262.1> - How to Perform a Healthcheck on Exalogic
<NOTE:1530781.1> - Exalogic Infrastructure Physical and Virtual Releases/PSUs – Software and Firmware Version Information
<NOTE:1995236.1> - Exalogic Patch Set Update (PSU) Release 2.0.6.2.1 (Linux - Virtual) for April 2015
<NOTE:1571367.1> - Exalogic Infrastructure PSU Upgrade - Known Issues
<NOTE:2078653.1> - Receiving Error When Patching Guest vServer with Exalogic PSU - ERROR: Missing "--uri URI-to-patch" option
<NOTE:1314535.1> - Exalogic Patch Set Updates (PSU) Master Note
<NOTE:1997466.1> - Information About Usage Of Exapatch Force ( -f or --force ) Option Used During Exalogic Patching
<NOTE:2031749.1> - FAQs Regarding vServer Patching Part Of Exalogic PSU Upgrade
<NOTE:2092033.1> - Exalogic Compute Node or Guest vServer PSU Upgrade Using Exapatch Fails With Error "IndexError: list index out of range"
<NOTE:2114516.1> - Exalogic X2-2 Compute Nodes ILOM Network Settings Lost After PSU Upgrade

Attachments

This solution has no attachment