Sun ZFS Storage 7420
 
Oracle ZFS Storage ZS5-2
 
Sun Storage 7110 Unified Storage System
 
Oracle ZFS Storage ZS3-2
 
Sun Storage 7210 Unified Storage System
 
Oracle ZFS Storage ZS4-4
 
Oracle ZFS Storage ZS5-4
 
Sun Storage 7410 Unified Storage System
 
Sun ZFS Storage 7120
 
Sun Storage 7310 Unified Storage System
 
Oracle ZFS Storage ZS3-4
 
Oracle ZFS Storage Appliance Racked System ZS4-4
 
Sun ZFS Storage 7320
 
Oracle ZFS Storage ZS3-BA
 

PLA-Support>Sun Systems>DISK>ZFS Storage>SN-DK: 7xxx NAS
 
_Old GCS Categories>Sun Microsystems>Storage - Disk>Unified Storage
 

In this Document

Symptoms

Changes

Cause

Solution

Generic information

VmWare

Solaris Client

Linux

Oracle database

References

Applies to:

Symptoms

In some circumstances, some NFS latencies might be discernible on NFS client side. This document provides some typical scenarios where NFS can be tuned to get the best performance.

Changes

Cause

Most of the tuning is to be done from client side. The next section introduces some best practices for VmWare as well as Oracle Database. Some of the information can be applied for any NFS client type.

Solution

Generic information

• Number of NFS concurrent connections:
  The only performance issue associated with NFS connections is having too few (a single one) from a single client trying to push 10Gbe Ethernet. Otherwise, do not worry about connections. In the implementation, we are "limited" to about 4 billion connections per system, at least in theory. In practice you're more likely to hit performance limits long before you get to that many - eg if each client produces 1 op/sec, you won't come anywhere close to 1 billion ops/sec. Additionally, you'll probably run into some physical memory limitations before you get very far into the millions of clients.

• Maximum Number of server threads:
  This should at least cover the number of concurrent NFS clients that is anticipated. Allowed range is 20 to 1000. The default value is 500, which is just a trade-off. This can be increased if you want to allocate more CPUs cycles to NFS protocol.

VmWare

• Filesystem block size:
  This is an important topic for best performance. For vSphere 4.x and recommended recordsize, see vmware-refarch-354079.pdf

• Number of guests per NFS share. There no performance improvements on increasing the number of NFS shares on the same pool. For snapshot/replication policies, it is logically easier to group some guests in some specific shares.

Solaris Client

• Parallelism:

To maximize parallelism between an NFS client and the ZFS storage appliance, the following entry can be added in /etc/system on client.

set rpcmod:clnt_max_conns = 8

As of S10u8, the default is 1, which allows for a single connection from the client to each server. This results in a single thread bottleneck for communication with each server. Increasing this value on the client increases the number of connections to the NAS head.

• Attribute caching on client side:

See Document 1164673.1 : NFS Performance Decline Introduced by Mount Option "actimeo=0". 

With actimeo=0, you disable attribute caching in the client, which can result in getattr storms during heavy file open/close activity.  With noac, you also disable client side write behind caching but still allow read caching (assuming forcedirectio is not also enabled). In Solaris, that turns all of your writes into synchronous 8KB writes to the server, which can have a dramatic performance impact. You should really never use actimeo=0 or noac unless explicitly required by an application. The only common application that should be using either of these is Oracle RAC. If you are given the choice of actimeo=0 or noac, choose actimeo=0.

Linux

Document 1646387.1 describes the impact of READDIRPLUS commands from Linux NFSv3 clients.

This slows down the performance when the file system is scanned by the commands like 'ls' or code that traverses the file system to get the attributes of the files for the first time.

Oracle database

There is a lot of conflicting information on this subject from a lot of different source.  The best reference of describing what mount options to use is Document 359515.1 : Mount Options for Oracle files when used with NAS devices

• wsize and rsize:
  In Document 359515.1, rsize and wsize are set to 32kB, which is conservative for a lot of NAS heads and network infrastructures. The good news is that 32kB always works, but the bad news is that another choice might work better. In the case of the ZFSSA, use a 128kB rsize and a 1024kBwsize for optimal performance on Q3.4 code and earlier. For 2011.Q1 (and later code) use 1024kB for both rsize and wsize.

For Appliance Firmware Release 2011.Q1 and later - use 1024KB for both rsize and wsize.

 

Also see Document ID 1495104.1 - Recommended Patches for Direct NFS Client

Back to Document 1331769.1 - Sun Storage 7000 Unified Storage System: How to Troubleshoot Performance Issues.

References

This solution has no attachment