FS System: FS1-2 Upgrade R6.x to R6.x Procedures

Asset ID:	1-71-1968129.1
Update Date:	2018-04-30
Keywords:

Solution Type Technical Instruction Sure

Solution 1968129.1 : FS System: FS1-2 Upgrade R6.x to R6.x Procedures

Applies to:

Oracle FS1-2 Flash Storage System - Version All Versions and later
Information in this document applies to any platform.

Goal

This document outlines the FS1-2 software upgrade steps.

Solution

To update code for FS1-2 software and firmware, there are two types of system upgrades: Disruptive Upgrade (DU) and Non-Distruptive Upgrade (NDU). DUs require an FS1-2 system restart, and thus IO must be stopped before attempting. And while NDUs do not require the cessation of IO, it is recommended that they be scheduled during non-peak IO periods.

Prerequisites/Planning

The following is a list of items detailing certain upgrade requirements.

Verify the version of software currently installed on the FS1-2 system.
- Using the Oracle FS System Manager GUI, from the System tab, select System Information in the navigation tree. The Software version will be displayed on the left side at the bottom of the System Information list.
- Using the CLI fscli, run: fscli version -list

Upgrade paths:

Upgrade Paths
Installed Release	Upgrade Path	Notes
6.1.3-0329.00 or below	Contact Oracle Support	Do not attempt an upgrade from version 6.1.3 and below without Oracle Support assistance.
6.1.4-0331.00 to 6.1.19-0410.01	DU to 6.2.3-0280.02	Please refer to the 6.2.3-280.02 Patch README for detailed DU instructions. If the current version is between R6.1.4 to R6.1.11 inclusive, it is recommended that the FS1 be shutdown prior to this upgrade. If the FS1 indicates that the "M3 Controller BIOS" is installed, select "ignore Compatibility" in order to perform this upgrade. If upgrading to 6.2.09-0353.00 (060209-035300) after the 6.2.3.0280.02 upgrade, it is not necessary to perform the Controller warm starts noted in the SAS HBA Firmware section of the 6.2.3-280.2 Patch README. If necessary, the system will automatically install the SAS firmware update as it upgrades to 6.2.9-0353.00. After the upgrade completes, verify system status, and continue to upgrade to 6.2.9-0353.00.
6.2.0-0253.00 to 6.2.3.280.2	DU to 6.2.9-0353.00	Please refer to the 6.2.9-353.00 Patch README for detailed DU instructions. After the upgrade completes, verify system status, and continue on the upgrade path below to 6.2.16-0545.01.
6.2.4.328.00 to 6.2.8-0348.00	DU or NDU to 6.2.9-0353.00	Please refer to the 6.2.9-353.00 Patch README for detailed upgrade instructions. After the upgrade completes, verify system status, and continue on the upgrade path below to 6.2.16-0545.01.
6.2.9-0353.00 to 6.2.15-0524.00	DU or NDU to 6.2.16-0545.01	The upgrade to 6.2.16-0545.01 may be performed disruptively or non-disruptively except for systems with iSCSI interfaces. They must be upgraded disruptively. DU: Enables support for Enhance Allocation for large systems and installs the BIOS/ILOM update. If the current version is between R6.2.9 and R6.2.11 inclusive, do NOT shutdown the system prior to the DU. If the current version is R6.2.12 or higher, a shutdown is recommended prior to the upgrade. NDU: Enhance Allocation will not be enabled and BIOS/ILOM files will not be updated.
6.2.16-544.00	See Instructions	Please refer to 6.2.16-0545.01 Patch README Special Notes for upgrading 6.2.16-544.00 systems.

Upgrades from R6.1.3 or lower are disruptive. Either before or after the upgrade is completed, please check the system's mode pages. See Document 1969899.1 FS System: How to Check the Drive Mode Page Setting on an FS1-2 System and resolve as needed.

NOTE: Oracle strongly recommends you upgrade to Release 6.2.11-0454-04 or later. Please see Customer Letter for FCO 380.

FS1-2 systems must be functioning normally (green) to be updated. If the system status is warning (yellow) or critical (red), these issues must be resolved before an upgrade can be attempted.
Any System Alerts must be resolved before attempting an upgrade.
For FS1-2 systems that require a Disruptive Upgrade and have MaxRep3 replication in their configuration, please see Document 2113569.1 FS System: Additional Steps Required for Disruptive Upgrade for Systems with MaxRep3.
Foreknowledge of the support (pillar) password - this can be a critical time saver if recovery actions are needed. Refer to Document 2046703.1 FS System: Passwords Associated with the Oracle FS1-2 Flash Storage System.

Preliminary Actions

The following actions must be completed before an upgrade can take place.

Compare this version to the current recommended release. See Document 1967797.1 FS System: How to Download Software and Firmware Updates for the FS1-2.
Review the Patch README for the version you want to install for any patch specific caveats. Also review the Release Notes for additional details on upgrades. Both are linked in the table of Document 1967797.1.
Review the Management Software Components Chapter of the Oracle Flash Storage System Administrator's Guide located at the Oracle Help Center under the Documentation Library.
If performing a Disruptive Upgrade (DU), confirm that all host IO has ceased:

Oracle FS System Manager GUI:

Select the SAN tab and expand Statistics and Trending in the navigation tree on the left if necessary.
Select LUNs
Confirm that IOPs for Reads and Writes are 0 for all LUNs.

Oracle FS System Manager CLI:

Create a list of the LUNs:
# fscli lun -list > lunlist.out
Use the lunlist.out file to view the performance statistics:
# fscli statistics -list -lun <lunlist.out > lun_perf_stats.out
Confirm that IOPs for Reads and Writes are 0 for all LUNs.

Upload a current log bundle to Oracle's Call-Home server or store it to a local system. See Document 1943289.1 FS System: How to Attach a Manual Log Bundle to an Existing MOS Service Request for details but ignore the steps to attach it to an existing Service Request if one is not already open. If there is a problem during the upgrade, this log bundle may be critical to the recovery of the FS1 system.
Do not view or collect Statistics and Trending information for at least 2 minutes before starting an Upgrade. After beginning the Software Update do not attempt to view any Reporting & Statistics screens.
Confirm the PMI (Private Management Interface) network connections between Controllers and Pilots are properly connected:

Pilot GUI:

From the System tab, select Hardware then Pilots
Double click on each Pilot
All interfaces must be green and Connected.

Pilot CLI:

# fscli pilot -list -details
...
EthernetConnection
     Port    : ETH1
     ConnectionReliability : CONSISTENT
     ConnectionStatus    : CONNECTED
EthernetConnection
     Port                  : ETH2
     ConnectionReliability : CONSISTENT
     ConnectionStatus    : CONNECTED
EthernetConnection
     Port                  : MANAGEMENT
     ConnectionReliability : CONSISTENT
     ConnectionStatus      : CONNECTED
EthernetConnection
     Port                  : SERIAL
     ConnectionReliability : CONSISTENT
     ConnectionStatus      : CONNECTED
EthernetConnection
     Port    : SERVICE_PORT
     ConnectionReliability : CONSISTENT
     ConnectionStatus      : CONNECTED
Controller GUI:

From the System tab, select Global Settings
Select Controller Ports
For both Controllers, Onboard Ethernet Ports 2 & 3 should be Connected.

Controller CLI, check Slot 0 Ports 2 & 3 from each Controller:

# fscli controller -list -port
....
/CONTROLLER-02
    Hba
        Slot : 0
            EthernetPort
                Number                    : 3
                Provisioning              : eth
                ConnectionStatus    : CONNECTED
                NegotiatedSpeed : 10000000000
                MaximumSpeed              : 10000000000
                IsPrivateInterconnectPort : true
                MacAddress                : 00:21:28:A1:C6:43
                SfpInformation
                    Status                : NOT_SFP_DEVICE
                    ConnectionType        : COPPER
                    Vendor
                  Name              :
                  PartNumber    :
                  Revision    :
                MtuBytes
                    Minimum     : 68
          Maximum : 9728
          EthernetPort
                Number                    : 2
                Provisioning    : eth
                ConnectionStatus          : CONNECTED
                NegotiatedSpeed           : 10000000000
          MaximumSpeed    : 10000000000
          IsPrivateInterconnectPort : true
          MacAddress                : 00:21:28:A1:C6:42
      SfpInformation
                    Status                : NOT_SFP_DEVICE
              ConnectionType        : COPPER
          Vendor
                  Name              :
              PartNumber        :
              Revision    :
      MtuBytes
                    Minimum : 68
            Maximum               : 9728

Stage the Software

The next step is to download the appropriate version of FS1-2 software from MOS and then stage or copy it to the FS1-2 Pilot. Once the software has been staged, it will remain on the Pilot until it is installed or the Pilots are rebooted. As such, it is perfectly fine to stage the software on one day and then perform the upgrade several days later.

Obtain the FS1-2 software. See Document 1968540.1 FS System: How to Find Software/Firmware Patches for the Oracle FS1-2 System in My Oracle Support.
- Download the software to a location accessible by the terminal that will perform the upgrade.
- Unzipping the patch will create a directory of the same name containing the software in Red Hat Package Manager format, an .rpm file.
Once the FS1-2 software has been downloaded from MOS, stage it to the active Pilot. This can be done using either the Oracle FS System Manager GUI or command-line fscli interfaces.

NOTE: To reduce the staging time needed, it is best to stage the software from a system that has a speedy network connection to the FS1-2 system. Doing this over a VPN connection can severely impact the time needed to stage the software.

GUI: Click on the Upload Software Package button. Once completed, the new software version will appear in the Staged Software Section:
CLI: the following commands will stage software version R6.1.6 to the FS1-2 system from a directory called \SOFTWARE and verify that the staging was successful:

# fscli software_update -add -package \SOFTWARE\OracleFS-SW-060106-034100.x86_64.rpm
Command Succeeded
# fscli software_update -validate -all newerVersionOnly
Command Succeeded

NOTE: if you see an "unsupported package combination" message after the Package Version in the Installed Software and Firmware section, please see KM Document 2046391.1 FS System: Oracle FS1-2 Flash Storage System May Report "Package version: unsupported package combination" After Firmware Upgrade for resolution.

Upgrade the FS1-2

Disable Call-Home. See Document 1942676.1 FS System: How to Disable Call-Home to Prevent Automatic Service Request (ASR) Generation. During the upgrade process components will be rebooted to enable the new versions of software/firmware being installed. If ASR is enabled, these reboots will generate spurious Service Requests. Temporarily disabling Call-Home will prevent this from happening.

NOTE: While all upgrades to R6.2.9 are disruptive, some also have specific recommendations as to whether the system should be shutdown or not. If the current version is R6.1.4 to R6.1.11 inclusive or R6.2.12 or higher, it is recommended that the system be shutdown before the upgrade begins. For systems running R6.2.11 do NOT shutdown the system prior either type of upgrade. Systems running R6.1.12 to R6.2.10 have no specific recommendation.

From the GUI:
Shut Down

From the CLI:

# fscli system -shutdown

Upgrade the FS1-2 software using GUI or CLI:

NOTE: Even an NDU will lose management access while the Pilots reboot. Pilots are not within the data path so this will not disrupt access to data.

From the GUI Support tab:

Select Software and Firmware in the navigation tree.
Click on Actions in the menu bar, and select Update Software.
For NDUs, be certain that the radio button in front of Update software without restarting system is selected.
For DUs, be certain that the radio button in front of Restart and upgrade software (disrupts data access) is selected. If the system has also been shutdown, be sure to check all of the boxes in the Software Update Options.
Press OK and Yes in the confirmation pop-up windows.

From the CLI:
1. Execute the following command for an NDU:
  
  #fscli software_update -install -all newerVersionOnly -noDisruptive
  command succeeded
2. If the system was not shutdown prior to upgrade, execute the following command for a DU:
  
  #fscli software_update -install -all newerVersionOnly -disruptive
  command succeeded
3. If the system was shutdown prior to the upgrade, execute the following command for a DU:
  
  #fscli software_update -install -all alwaysInstall -disruptive -forceControllerShutdown -ignoreBadSystemState -ignoreCompatibilityChecking -ignoreExistingAlerts -ignoreOperationPoolDraining -overridePreviousFailedUpdate
The amount of time needed for an upgrade depends on system size. A typical upgrade will take 20-30 minutes. Its progress can be monitored by observing the tasks (button in lower right corner of GUI):

Verify the Upgrade

After the Pilot reboot has completed, confirm the upgrade was successful by checking the Events Log:

Confirmation of Upgrade

How to Update Disk Drive Firmware

NOTE: While upgrading the Oracle FS System Manager software does not require an outage, the drive firmware upgrade does. As such, customers may wish to consider taking a single outage. System must have a Normal status with no System Alerts prior to starting any upgrade.

NOTE: It is best to take advantage of the existing outage to update all drive firmware versions. Doing so will also prepare the FS1-2 for future Enclosure add-ons to be upgraded before they are put into production. It is also necessary to avoid "Unsupported" drive messages when replacing failed drives where the replacement has a higher version than the existing drives.

Using the GUI to update Drive Enclosure drives:

Select Support tab
Select Software and Firmware in the navigation tree
From the menu bar select Actions and then Update Drive Firmware or from the main window, right click and select Update Drive Firmware
In the resultant pop-up window check all the boxes in the Install? column and then OK
Confirm the upgrade will be disruptive
System will do a Cold Start

NOTE: after this initial Cold Start, the Controllers will reboot to a certain point in ConMan and then download the actual drive firmware. After which a Cold Start Failed message will appear and a second Cold Start will take place. This is normal.

Using the CLI to upgrade Drive Enclosure drives:
1. The following command will confirm what software version is staged:
  
  C:\>fscli software_update -list
  Staged Software: /STAGED
  Id : 414B303031323639A13A000000000001
  PackageVersion : 2090-00001-060200-025100
  ...
  InstalledDriveFirmware
  DriveFirmware
  DriveType : SanDisk 400GB SSD
  Version : 2052-00004-060000-000400
  DriveFirmware
  DriveType : Sunset Cove Plus 400GB SSD
  Version : 2052-00006-060100-026900
  DriveFirmware
  DriveType : Hitachi 900GB HDD
  Version : 2052-00002-060100-009700
  DriveFirmware
  DriveType : Sunset Cove Plus 1600GB SSD
  Version : 2052-00007-060100-026900
  DriveFirmware
  DriveType : SanDisk 1600GB SSD
  Version : 2052-00005-060000-000400
  DriveFirmware
  DriveType : HGST AriesHC 8TB (non-FIPS) HDD
  Version : 2052-00008-060200-022800
  DriveFirmware
  DriveType : Hitachi 300GB HDD
  Version : 2052-00001-06000-009700
  DriveFirmware
  DriveType : Hitachi 4TB HDD
  Version : 2052-00003-060200-004900
  
  Matching the PackageVersion at the top with the with the DriveFirmware Versions toward the end will indicate which version need to be installed.
2. Each drive firmware version will need to be installed individually, This example shows an upgrade the H109030SESUN300G or 300GB drives:
  
  C:\>fscli software_update -install -drive "Hitachi 300GB HDD"
  Warning: Contact the Support Center before proceeding! Proceeding without their assistance could risk data loss.
  Continue(y/N)? y
  
  Command Succeeded
  C:\>
3. System will do a Cold Start.
  
  NOTE: after this initial Cold Start, the Controllers will reboot to a certain point in ConMan and then download the actual drive firmware. After which a Cold Start Failed message will appear and a second Cold Start will take place. This is normal.
  .

Post Upgrade Actions

Once the upgrade has completed, it is necessary to upgrade the software to manage the FS1-2 system. Point a browser at the shared IP address of the FS1-2 system to access the new version of the software:
Download and Install the new GUI for your platform, reference Document 2085329.1 FS System: How to Download and Install the Oracle FS System Manager (GUI) client.
Download and Install the new CLI for your platform, reference Document 1991938.1 FS System: How to Obtain and Install the fscli Tool Software.
Re-enable Call-Home, reference Document 1942676.1 FS System: How to Disable Call-Home to Prevent Automatic Service Request (ASR) Generation.

References

<NOTE:1969899.1> - FS System: How to Check the Drive Mode Page Setting on an FS1-2 System
<NOTE:1967797.1> - FS System: How to Download Software and Firmware Updates for the FS1-2
<NOTE:1943289.1> - FS System: How to Gather and Attach a Manual Log Bundle to an Existing MOS Service Request
<NOTE:2120675.1> - Flash Storage System FS1-2 SAS Driver Failures May Lead to Loss of Data Access
<NOTE:1968540.1> - FS System: How to Find Software and Firmware Patches for the Oracle FS1-2 System in My Oracle Support
<NOTE:1544005.1> - My Oracle Support 'How-to' Series: How to Validate your Access and Create, Manage, and Gain Management Attention to Hardware, Software or Cloud Service Requests
<NOTE:1942676.1> - FS System: How to Disable Call-Home to Prevent Automatic Service Request (ASR) Generation
<NOTE:2046391.1> - FS System: Oracle FS1-2 Flash Storage System May Report "Package version: unsupported package combination" After Firmware Upgrade

Attachments

This solution has no attachment