Sun ZFS Storage 7420
 
Sun Storage 7110 Unified Storage System
 
Oracle ZFS Storage ZS3-2
 
Oracle ZFS Storage ZS4-4
 
Sun Storage 7210 Unified Storage System
 
Sun Storage 7410 Unified Storage System
 
Oracle ZFS Storage ZS3-4
 
Sun Storage 7310 Unified Storage System
 
Sun ZFS Storage 7120
 
Sun ZFS Storage 7320
 
Oracle ZFS Storage ZS3-BA
 
Sun Storage 7720 Unified Storage System
 

PLA-Support>Sun Systems>DISK>ZFS Storage>SN-DK: 7xxx NAS
 
_Old GCS Categories>Sun Microsystems>Storage - Disk>Unified Storage
 

This document is a resolution path for the general issues faced during firmware upgrades on ZFSSA or Amber Road.

In this Document

Purpose

Troubleshooting Steps

General Recommendations:

Health Check:

If a problem occurs:

Other References:

References

Applies to:

Purpose

This document discusses general resolutions for issues faced during firmware upgrades on ZFS Storage Appliance (Unified storage 7000 series).

Additionally the document also contains links and references to other documents required to troubleshoot specific issues.

Troubleshooting Steps

The Appliance Kit or ak software contains a customized OS image which also includes the firmware for hardware components like disks, SIM, SAS HBA, Logzilla, disks etc.

Currently the SP firmware and BIOS need to be upgraded separately and are not part of the Appliance Kit software. See Document 1174698.1 to check the SP and BIOS levels.  If these are below the recommended versions please open a Service Request with Oracle Support to have them upgraded.  The SP and BIOS upgrade can be carried out remotely by an Oracle Support representative using Shared Shell or some other remote access technology to access the Service Processor (SP / ILOM).

Since the firmware upgrade is performed online we should have two paths available to the disks and all SIMs should be online. Most of the time it is seen that existing issues on the system may prevent the upgrade of the firmware and thus these need to be resolved before proceeding further..

NOTE: Disk components like read cache ssds, write cache ssds and data disk drives that are not part of a current active zpool will not be upgraded with new firmware. Also the array may show pending firmware upgrades for these devices. This is normal array operation. In order to get these devices upgraded they need to be added to an existing zpool. Once they are part of a zpool they will have their firmware checked and upgraded as necessary.

General Recommendations:

To avoid issues during a firmware upgrade please check for the following :

• Access to ILOM, BUI/CLI interface IP (private IP)

• No resilvering is going on

• Check and resolve existing alerts and problems which may prevent update

• Previous Firmware upgrades are not in progress i.e. no pending updates

• In a clustered configuration do not perform failback before firmware updates are completed.
  You can check the status as:
   BUI: Maintenance-> System Updates
   CLI: Pre OS 8.6: maintenance-> system -> updates-> firmwareUpdatesDetails
          8.6 and later: maintenance-> system -> updates-> firmware-> show

• Check the most recent product release notes for additional preconditions that should be observed for the software release to which you are upgrading.

  Check Oracle ZFS Storage Appliance: Software Updates (Doc ID 2021771.1) for Firmware Release notes.

Health Check:

Though a pre-install health check is performed automatically as part of the upgrade, it is always recommended to perform a manual check to identify any existing issues with the system prior to beginning the upgrade. Specifically check for the following:

• No other active problems including software or hardware

• Physical inspection of hardware for any active problems including attached JBODs

As mentioned in the pre-requisites please be sure that no hardware issues persist on the head and the attached JBODs. You also need to verify the health of the peer node in the case of a cluster.

If a problem occurs:

Define the problem and try to isolate the issue, identify at what step the firmware upgrade fails. Collect the messages and symptoms to proceed further.

Depending upon the issue faced follow the troubleshooting steps valid for your environment mentioned below.

• Failed Health Check : Please see Document 1401522.1

• Failed hardware : In case a hardware fault is found, please raise a service request with the latest support bundle to get the hardware diagnosed and replaced.
  See <Document 1019887.1> for details on how to collect a Support Bundle.

• Check for availability of space in the system pool. The system log messages may be helpful here. Check the BUI:

  Maintenance - LOGS - SYSTEM

On a live system check the /var/ak/logs/jobs.txt file for details about the failed upgrade, this log can be found as logs/jobs.txt in a Support Bundle.

The /var/ak/logs/debug.sys will also be useful, in a Support Bundle this file is logs/debug.sys

On clustered system, if checking the cluster status like 'configuration cluster show' hangs it could be that there are pending HW updates pending.  This can be checked by doing a gcore of the akd process (gcore `pgrep -xo akd`) and then doing the following:

# mdb core.123

> ::ak_rm_elem -v !grep -c UPDATE_NEEDED
269

There were still items being pushed out.  Once the HW updates are complete, then the command operated correctly.

Other References:

Oracle ZFS Storage Appliance Customer Service Manual (Release OS8.7.x) - http://docs.oracle.com/cd/E79446_01/pdf/E79459.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release OS8.7.0) - http://docs.oracle.com/cd/E78901_01/pdf/E78914.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release OS8.6.x) - http://docs.oracle.com/cd/E76475_01/pdf/E76485.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release OS8.6.0) - http://docs.oracle.com/cd/E71909_01/pdf/E71921.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release 2013.1.5.0) - http://docs.oracle.com/cd/E69262_01/pdf/E69256.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release 2013.1.4.0) - http://docs.oracle.com/cd/E56047_01/pdf/E56078.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release 2013.1.3.0) - http://docs.oracle.com/cd/E56021_01/pdf/E55850.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release 2013.1.2.0) - http://docs.oracle.com/cd/E51475_01/pdf/E52874.pdf

Oracle ZFS Storage Appliance Customer Service Manual (Release 2013.1.0) - http://docs.oracle.com/cd/E27998_01/pdf/E48492.pdf

Sun ZFS Storage 7120, 7320, and 7420 Appliance Customer Service Manual (Release 2011.1.5) - http://docs.oracle.com/cd/E28317_01/pdf/E38247.pdf

Sun Storage 7000 Unified Storage System Service Manual - http://docs.oracle.com/cd/E19548-01/pdf/821-1388.pdf

MOS Document ID  2021771.1  -  Oracle ZFS Storage Appliance: Software Updates

At this point, if you have validated that if you have followed pre-requisites and health checks and the symptoms mentioned above doesn't match in your environment and the problem still exists and further  troubleshooting is required please contact Oracle Support Services to raise a new Service Request (SR) via the My Oracle Support portal (https://support.oracle.com) or by telephoning your Oracle Global Support telephone number.

Back to Document 1416406.1 Sun ZFS Appliances Troubleshooting Resource Center.

Currency check  03-JUL-2015

References

This solution has no attachment