FS System: Handling the Performance Impact from Integrity Checks

Asset ID:	1-79-2044517.1
Update Date:	2016-04-12
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 2044517.1 : FS System: Handling the Performance Impact from Integrity Checks

Applies to:

Oracle FS1-2 Flash Storage System - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Purpose

To understand the impact of an Auto-tier IC (Integrity Check) on Lost Data or Volumes Have Lost Mappings alerts.

Usually, the IC has no impact on processes that are running on an Oracle FS System. However, if an administrator responds to system alerts about lost data or lost mappings while an IC is running, the IC continues without interruption. Then, after the IC completes, the Oracle FS System can act on the administrator’s request to clear lost data and clear lost maps.

Scope

This information applies to administrators of the Oracle FS System who manage the Storage Domains.

Details

If the Oracle FS System detects a problem with the address location on Auto-tiered LUNs, the system sends a "Volumes Have Lost Mappings" system alert. Then, the system immediately starts to run an IC on the Storage Domain. Checks started by the system cannot be cancelled and can take several minutes to several hours to complete, depending on the number of LUNs and the amount of data that the system checks.

The system also automatically runs the IC in the following circumstances:

After a restart operation resulting from the system not shutting down cleanly.
After a Controller failover.
After a warmstart, if the administrator does not respond to a Lost Data system alert.
After a warmstart, if the system did not complete its data mappings or data migrations.
After detecting a media error on a metadata volume. The media error can also result in a Volumes Have Lost Mappings alert but the system does not wait for your response before proceeding with the IC.

If the host attempts to access data when a Volumes Have Lost Mappings alert has been issued, then the system sends a media error to the host. More information about ICs can be found in the topic “Storage Domain Integrity Check” in Oracle Flash Storage System Administrator's Guide. See References for more information.

How to Check the Integrity of a Storage Domain

If you suspect that Auto‑tier data might be lost or corrupted, you can run an IC on a Storage Domain any time. If you initiate an IC while the system is already performing the check, the system ignores your request.

The Oracle FS System automatically runs the IC when the system has encountered a Volumes Have Lost Mappings system alert.

Navigate to System > Storage Domains.
Select the Storage Domain on which to run the integrity check.
Select Actions > Verify Storage Domain.
The Storage Domain status changes to Background Activity.
Click OK.

When the IC is complete, the Storage Domain status changes to Normal. You can monitor the process by clicking Tasks on the status bar.

Next Steps

After clearing the host mappings, the administrator should verify their data using host-based tools such as fsck. Otherwise, the host might read an affected block and cause the Oracle FS System to return zero (0) to the host, which might require the administrator to restore the system from backups.

How to tell if an IC is already running

Run the FSCLI command "fscli task -list -details" against the FS system in question. The output is below.

/CmBackgroundTypeSlatIntegrityCheckerCmTask/
Name : CmBackgroundTypeSlatIntegrityCheckerCmTask
TaskId : *
Status : IN_PROCESS
PercentComplete : 47.00
StartTime : 2016-03-24T00:16:00.423
Username : SYSTEM
CreatedTime : 2016-03-24T00:16:00.423
CompletedTime :
EstimatedCompletionTime : 2016-03-25T02:30:17.693

The pilot logs can also be queried by running a grep command on the server.log files.

grep IntegrityChecker server.log.*
server.log.*:CmBackgroundTypeSlatIntegrityCheckerCmTask 002:10:50 52% IN_PROCESS /CmBackgroundTypeSlatIntegrityCheckerCmTask/

How to Cancel the IC

If the IC that you initiated is taking too long, or if the process impacts system performance, you can cancel the process by highlighting the appropriate Storage Domain and then selecting Actions > Cancel Verify Storage Domain.

Cancelling the IC does not affect the system or data. You cannot cancel an IC that is triggered by a Volumes Have Lost Mappings alert.

How to Change the Process Priority

If the impact of the IC process I/O is a concern, then the administrator can set the Background Process Priority to Minimize Impact.

If there is minimal or no host I/O activity, then the administrator can set the Background Process Priority to Maximize Speed to enable the IC to run faster.

Changing the Storage Domain background priority might impact other background processes on the system, such as tasks involving Drive Enclosure zeroing.

REFERENCES:

See sections on “Storage Domain Integrity Check,” “Check the Integrity of a Storage Domain,” and “Update the Background Process Priority” in Oracle Flash Storage System Administrator's Guide, which is available in the Oracle Flash Storage System Documentation Library at http://docs.oracle.com/en/storage/#fla.

Attachments

This solution has no attachment