Oracle ZFS Storage Appliance: Using ZFS Deduplication

Asset ID:	1-79-1996349.1
Update Date:	2018-05-30
Keywords:

Solution Type Predictive Self-Healing Sure

Solution 1996349.1 : Oracle ZFS Storage Appliance: Using ZFS Deduplication

Applies to:

Sun ZFS Storage 7420 - Version All Versions and later
Oracle ZFS Storage ZS3-BA - Version All Versions and later
Sun Storage 7110 Unified Storage System - Version All Versions and later
Oracle ZFS Storage ZS3-4 - Version All Versions and later
Oracle ZFS Storage ZS3-2 - Version All Versions and later
7000 Appliance OS (Fishworks)

Purpose

For ZFSSA, we do not recommend that customers use deduplication in any situation.

Scope

This document is intended to help customers configuring their ZFS appliance and the properties of their pool.

Details

Deduplication is a feature of ZFS, that is available on Solaris and on the ZFSSA.

For ZFSSA, we do not recommend that customers use deduplication in any situation.

The reason is the size of the pools : as pools are huge, DDT (deduplication table) metadata entries may get huge as well.

And the current implementation of ZFS deduplication does not handle huge DDT metadata entries.

There is currently no way to limit the size of the DDT metadata table which references all deduplicated blocks. As a consequence, there is a risk that this table grows and that we start writing DDT metadata blocks on disks instead of keeping them in memory.

As we update the DDT metadata from disks, we may be affected by small IOs being done within the sync phase, which may cause a substantial performance impact.

Development Engineering is currently working on improving the way DDT works on the ZFSSA.

At the moment, the use of deduplication on the ZFSSA pools is deprecated on the ZFSSA.

For customer who have already set deduplication on their pool, it is advised to turn off deduplication, using the BUI or CLI in the 'shares' menu.

To do that, select the project or share which has deduplication property checked - and uncheck it.

From the CLI :

> shares select project set dedup=false

This action will prevent new blocks from being deduplicated.

If the data from the pool is being modified regularly, we will progressively decrease the use of deduplication from the ZFSSA, with this setting in place.

Customers who are experiencing performance issues and who have, or have had, deduplication in place on the ZFSSA, should open a service request and involve Oracle Support to get the ZFS deduplication status on their ZFSSA examined.

As the online help recommends, to determine if performance has been adversely affected by deduplication, enable "Advanced analytics" and then use analytics to measure "ZFS DMU operations broken down by DMU object type" and check for a higher rate of sustained DDT operations (Data Duplication Table operations) as compared to ZFS operations.

If this is happening, more I/O is being used for serving the deduplication table (DDT) rather than file I/O.

ZFS DMU operations per second broken down by DMU object type

Deduplication changes in ZFS Appliance Firmware Release OS 8.7.x

Deduplication 2.0

The ZFS implementation is inline, block-level deduplication, which means that redundant data can be reduced when it is written to disk.

The way it works is that a hash is calculated for each block written to storage, which is then compared to entries in a deduplication table (DDT) of previously written blocks. When an incoming block matches, it is not written to storage. Instead, a reference pointer to the existing block is used, resulting in reduced storage space for large amounts of backup data. Deduplication 2.0 is targeted specifically for backup use cases. Full or level 0 backups yield the highest rate of deduplicated data.

Benefits

The new design provides that the DDT table is available in memory,and if the size of the table increases, it is available on DDT solid-state drives (SSDs), leveraging the existing Hybrid Storage Pool model to scale performance. Hybrid Storage Pool is a feature of Oracle ZFS Storage Appliance.

Overall deduplication 2.0 benefits include the following:

Provides excellent data reduction yields for backup use cases
Leverages powerful Hybrid Storage Pool for scalable deduplication performance
No additional licensing costs for deduplication or compression

Recommended configuration practices for the above backup products are summarized below and will be covered more specifically in separate white papers.

More detailed white papers and virtualization product information will be available at:

https://community.oracle.com/community/server_%26_storage_systems/storage/zfs-storage-appliance

Oracle ZFS Storage Appliance Requirements

Oracle ZFS Storage Appliance deduplication 2.0 is supported with the following hardware and software requirements:

OS8.7 software is required but no separate deduplication license is required.
The following platforms are supported and include recommended memory requirements:
- Oracle ZFS Storage Appliance ZFS4-4 with at least 1 TB of memory per controller
- Oracle ZFS Storage Appliance ZS5-2 with at least 384GB of memory per controller
- Oracle ZFS Storage Appliance ZS5-4 with at least 1.50 TB of memory per controller
3.2 TB SSDs for meta devices (DDT SSDs) and two for every four trays are recommended
Oracle Storage Drive Enclosure DE3-24C storage trays for meta devices (DDT SSDs) are required.
Oracle ZFS Storage Appliance 7x20 and later models using deduplication 1.0 are supported with deduplication 2.0 and no SSD is required:
- Upon upgrade to OS8.7, the existing deduplication table is automatically migrated to the new format.

There are 3 sets of analytics used to monitor performance of deduplication:

ZFS DMU operations (by DMU object type)
- This analytic will show you how many operations are being performed against the Data Deduplication Table compared to other ZFS operations.
Meta device bytes used (by pool)
- Amount of space used on the metadata devices. This statistic will remain blank until at least 1% of the meta device capacity is used.
Meta device percent used (by pool)
- Percent of space used on the metadata devices. This statistic will remain blank until at least 1% of the meta device capacity is used.

Please note: Performance issues can occur after deferred updates are applied to a system using dedup (version 1) after an upgrade to OS 8.7.x

See MOS Doc ID 2306637.1 (Oracle ZFS Storage Appliance: Performance Issue during Dedupv1 to Dedupv2 Migration after upgrade to OS8.7)

***Checked for relevance on 30-MAY-2018***

Attachments

This solution has no attachment