Asset ID:	1-72-1983170.1
Update Date:	2018-01-05
Keywords:

---

Solution Type  Problem Resolution Sure

Solution  1983170.1 :   Oracle ZFS Storage Appliance: Slow NDMP ZFS Backup Performance  

In this Document

Symptoms

Changes

Cause

Solution

References

---

Created from <SR 3-9031282681>

Applies to:

Sun ZFS Storage 7420 - Version All Versions to All Versions [Release All Releases]
Oracle ZFS Storage ZS3-4 - Version All Versions to All Versions [Release All Releases]
Oracle ZFS Storage ZS3-2 - Version All Versions to All Versions [Release All Releases]
Sun ZFS Storage 7120 - Version All Versions to All Versions [Release All Releases]
Sun ZFS Storage 7320 - Version All Versions to All Versions [Release All Releases]
7000 Appliance OS (Fishworks)

Symptoms

Slow ZFS type NDMP backup performance.

 

Changes

Upgraded to 2013 firmware code.

2011 firmware code was able to push around 60-80MB/sec but dropped to 16-25MB/sec in 2013 firmware code.

Cause

Prefetch data is evicted before send thread sending the data.

 

Specific workload where FC luns are used by Informix database with 2K block I/O size.

Use dtrace script timebytes20.d during NDMP backup to determine the ZFS and NDMP activities.

Solution

Engage Oracle support by opening a Service Request.

 

Workaround is to set zfs_arc_p_min_shift to a value of 3 which changes the MRU size from 1/16 to 1/8.

Zfs_arc_p_min_shift tuning can be done using tuning_zfs_arc_p_min_shift.akwf.

System needs to be rebooted for the change to take affect.


Known bugs that could have affected this behaviour are still pending:

• 17765840 l2 headers should not be counted in arc_size
• 17855980 arc_shrink should enforce a minimum value for arc_p
• 17856080 arc_meta_used is not necessarily all MRU

References

<BUG:17765840> - L2 HEADERS SHOULD NOT BE COUNTED IN ARC_SIZE
<NOTE:1900085.1> - Oracle ZFS Storage Appliances: NDMP Backup Performance Degradation (type=zfs) after 2013 Firmware Code Upgrade
<BUG:19232815> - ZFS APPLIANCE NDMP BACKUP PERFORMANCE IS SLOW AFTER UPGRADING TO 2013
<BUG:18820914> - NDMP BACKUPS UP TO 40% SLOWER ON AK8 VS 2011.1.6 CODE
<NOTE:1395374.1> - Sun Storage 7000 Unified Storage System: Troubleshooting NDMP issues on the ZFS Storage Appliance
<NOTE:1388017.1> - Sun Storage 7000 Unified Storage System: NDMP backup/restore performance
<BUG:17856080> - ARC_META_USED IS NOT NECESSARILY ALL MRU
<BUG:17855980> - ARC_SHRINK SHOULD ENFORCE A MINIMUM VALUE FOR ARC_P

Attachments

This solution has no attachment