Sun Storage 7000 Unified Storage System: How to add L2ARC cache SSDs (Readzillas) to a pool

Asset ID:	1-71-1333120.1
Update Date:	2018-05-10
Keywords:

Solution Type Technical Instruction Sure

Solution 1333120.1 : Sun Storage 7000 Unified Storage System: How to add L2ARC cache SSDs (Readzillas) to a pool

Applies to:

Sun ZFS Storage 7320 - Version All Versions and later
Sun ZFS Storage 7420 - Version All Versions and later
Sun Storage 7720 Unified Storage System - Version All Versions and later
Sun Storage 7410 Unified Storage System - Version All Versions and later
Sun Storage 7310 Unified Storage System - Version All Versions and later
7000 Appliance OS (Fishworks)

Goal

Purpose

An introduction to the L2ARC read cache Solid State Drive devices known as Readzillas in the ZFS Storage Appliance 7000 series, and procedures for adding them to pools.

Scope

Intended for all storage administrators who wish to configure their ZFS Storage Appliance 7000 series systems with Readzillas to improve read performance for certain I/O workloads.

Starting in Appliance Firmware Release AK 8.6.0 (2013.1.6.0), administrators can remove read cache drives from pools using the BUI or CLI.

From the BUI online help: Removing a Cache or Log Device from an Existing Storage Pool (BUI)

Use the following task to remove a read cache or log device from an existing storage pool.

Before You Begin

Do not perform a pool configuration operation while a disk firmware upgrade is occurring. To check if an upgrade is in progress, navigate to Maintenance > System.

1. Go to Configuration > Storage.
2. From the Available Pools list, select an online pool from which to remove the device.
3. Click REMOVE.
4. Select the number of log and cache devices to be removed from the storage pool.

Note - If the log devices use a mirrored profile, a message reminds you to select an even number of log devices to remove. If they use a striped profile, you may remove an even or odd number of devices.

5. Click COMMIT.

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - Disk Storage ZFS Storage Appliance Community

Solution

What is a Readzilla

Readzillas are low latency Solid State Drives which can be used to provide an additional layer of read caching between the ARC in main memory and the disks for a particular zpool. This intermediate layer of read cache is called the L2ARC. Using Readzillas will provide the greatest performance benefit for read-heavy workloads where the working set is fairly static and is too large to be cached into the main ARC. Because they are low latency devices reading data cached in the L2ARC will be much faster than reading data directly from disk. Readzillas can only be used in the 7310, 7410, 7320, 7420 and 7720 ZFS Storage Appliance systems.

Configuration in a zpool

Readzillas are associated with a particular zpool as cache devices, either when the pool is first created or they can also be added easily at a later date. The readzilla devices are inserted into the drive slots in the NAS head, so in the case of a cluster, they can only be associated with a zpool controlled by that particular head. Once they have been associated with a zpool then that zpool can only use the readzillas when it is being mastered by the head that contains those readzillas. If a takeover occurs and the zpool switches heads then it cannot use the readzillas in the original head. It is possible to add readzillas on the second cluster head to the same zpool so that a zpool will have access to read cache whatever head it is on. However those readzillas will then need to "warm up" again after a takeover by having the most recent and most frequently accessed data read into them which may take several hours. This configuration will also mean that one head will always have readzillas that are not being used which may be seen as an uneconomical solution. Since it nay take several hours for read cache to reach optimal levels it may be better to simply fix whatever caused the takeover in the first place and failback the pool to use the readzillas on the original head again. Readzillas should never be removed from one head and transferred to the partner in an attempt to provide cache devices for a pool which has been taken over. This will not work.

When readzillas are added to a pool they are simply added as a pool of cache - they cannot be added as a mirror or raidz configuration. If there is an I/O failure to a readzilla then that read is tried again directly from disk. If one readzilla is lost the data in any other readzilla that was part of the zpool cache is still available. The cached data in the failed readzilla is lost but this is not real data loss since any failed I/O to the readzilla is simply retried from disk.

Adding Readzillas

To add a readzilla:

Physically insert the readzilla in an appropriate slot. Once the drives have been physically installed you should wait for a couple of minutes for the software to pick up and recognise the new devices.

Then add the new readzilla to pool-0 using either the CLI or BUI methods detailed below:

CLI

nas-1a:> configuration storage

Ensure that you are in the correct pool, in the example below there was only one pool - pool0 which is now shown as the selected pool in brackets.
nas-1a:configuration storage (pool-0)>
nas-1a:configuration storage (pool-0)> add

This should now give you the prompt
nas-1a:configuration storage (pool-0) verify>

You can verify that you can see the readzilla cache device now by using "show"
nas-1a:configuration storage (pool-0) verify> show

This should show a summary of the devices seen - your readzilla(s) should be visible under the CACHE heading.
If this all looks OK simply type "done" to acknowledge the "verify".
nas-1a:configuration storage (pool-0) verify> done

Now type " done" again to finish the operation and add the readzilla cache device(s).

BUI

Click through to the "Configuration -> Storage" screen.

Under "Available Pools" select the correct pool and then click on the "ADD" button.

This will take you to the next screen which should show the readzilla(s) under "Cache Devices"

Provided this looks correct click on "COMMIT" to add the readzilla(s)

See the full example below:

dwstor1:> configuration storage
dwstor1:configuration storage (Oracle)> ls
Pools:

      POOL             OWNER         DATA PROFILE LOG PROFILE   STATUS
pool = Oracle           dwstor1       mirror        log_mirror    online
      Oracle1          dwstor2       -             -             exported

Properties:
                         pool = Oracle
                       status = online
                        owner = dwstor1
                      profile = mirror
                  log_profile = log_mirror
                cache_profile = cache_stripe
                        scrub = resilver completed after 0h0m with 0 errors
dwstor1:configuration storage (Oracle)> add

Verify and add devices. Devices may be added on a per-device basis, however
SATA devices in SAS-1 enclosures may be added in half- or whole-chassis units
only. While affected devices may be added, they will not be available for use
without reconfiguring the pool; for best results, defer configuring storage
until any problems can be repaired. Mixing devices of differing speeds within a
storage pool is strongly discouraged.

Subcommands that are valid in this context:

help [topic]         => Get context-sensitive help. If [topic] is specified,
                          it must be one of "builtins", "commands", "general",
                          "help", "script" or "properties".

show                 => Show information pertinent to the current context

abort                => Abort this task (potentially resulting in a
                          misconfigured system)

done                 => Finish operating on "verify"

get [prop]           => Get value for property [prop]. ("help properties"
                          for valid properties.) If [prop] is not specified,
                          returns values for all properties.

set [prop]           => Set property [prop] to [value]. ("help properties"
                          for valid properties.) For properties taking list
                          values, [value] should be a comma-separated list of
                          values.

dwstor1:configuration storage (Oracle) verify> show

       ID     TATUS   ALLOCATION   DATA          LOG           CACHE       RPM
       --    -------   ----------   -----------   -----------   ----------- -----
        0         ok       custom   0             0             1/1   93.2G
                                    -----------   -----------   -----------
                                    0             0             1     93.2G

dwstor1:configuration storage (Oracle) verify> done

Choose Storage Profile. Configure available storage into a pool by defining its
underlying redundancy profile. Carefully read the profile descriptions to
understand how each balances the inherent trade-offs between availability,
performance, and capacity, and select the profile that best fits your workload.
If available, NSPF indicates no single point of failure, which affords certain
profiles the ability for a pool to survive through loss of a single disk shelf.

Subcommands that are valid in this context:

help [topic]         => Get context-sensitive help. If [topic] is specified,
                          it must be one of "builtins", "commands", "general",
                          "help", "script" or "properties".

show                 => Show information pertinent to the current context

abort                => Abort this task (potentially resulting in a
                          misconfigured system)

done                 => Finish operating on "config"

get [prop]           => Get value for property [prop]. ("help properties"
                          for valid properties.) If [prop] is not specified,
                          returns values for all properties.

set [prop]           => Set property [prop] to [value]. ("help properties"
                          for valid properties.) For properties taking list
                          values, [value] should be a comma-separated list of
                          values.

dwstor1:configuration storage (Oracle) config> done
dwstor1:configuration storage (Oracle)> cd /

TSC Engineer also noted that the following (CLI) command may be required: 'set 0-cache=1'

After the 'configuration storage add' the following is needed: `set 0-cache=1`. Without it, the following is reported:

NAS:configuration storage verify> done
You must have at least one data device to configure storage

Example:

NAS:> configuration storage add

Verify and add devices. Verify that storage is correctly attached and
functioning. If devices are missing or malfunctioning, they will not be
available for use and cannot be added without reconfiguring the pool. It is
recommended that you fix any problems before configuring storage on the
appliance. Mixing device types and speeds is strongly discouraged.

Subcommands that are valid in this context:

   help [topic]         => Get context-sensitive help. If [topic] is specified,
                           it must be one of "builtins", "commands", "general",
                           "help", "script" or "properties".

   show                 => Show information pertinent to the current context

   abort                => Abort this task (potentially resulting in a
                           misconfigured system)

   done                 => Finish operating on "verify"

   get [prop]           => Get value for property [prop]. ("help properties"
                           for valid properties.) If [prop] is not specified,
                           returns values for all properties.

   set [prop]           => Set property [prop] to [value]. ("help properties"
                           for valid properties.) For properties taking list
                           values, [value] should be a comma-separated list of
                           values.

NAS:configuration storage verify> show

        ID     STATUS   ALLOCATION   DATA          LOG           CACHE       RPM
        --    -------   ----------   -----------   -----------   ----------- -----
         0         ok       custom   0             0             0/1   477G
                                     -----------   -----------   -----------
                                     0             0             0

NAS:configuration storage verify> set 0-cache=1
                       0-cache = 1
NAS:configuration storage verify> show

        ID     STATUS   ALLOCATION   DATA          LOG           CACHE       RPM
        --    -------   ----------   -----------   -----------   ----------- -----
         0         ok       custom   0             0             1/1   477G
                                     -----------   -----------   -----------
                                     0             0             1     477G

NAS:configuration storage verify> done

Choose Storage Profile. Configure available storage into a pool by defining its
underlying redundancy profile. Carefully read the profile descriptions to
understand how each balances the inherent trade-offs between availability,
performance, and capacity, and select the profile that best fits your workload.
If available, NSPF indicates no single point of failure, which affords certain
profiles the ability for a pool to survive through loss of a single disk shelf.

Subcommands that are valid in this context:

   help [topic]         => Get context-sensitive help. If [topic] is specified,
                           it must be one of "builtins", "commands", "general",
                           "help", "script" or "properties".

   show                 => Show information pertinent to the current context

   abort                => Abort this task (potentially resulting in a
                           misconfigured system)

   done                 => Finish operating on "config"

   get [prop]           => Get value for property [prop]. ("help properties"
                           for valid properties.) If [prop] is not specified,
                           returns values for all properties.

   set [prop]           => Set property [prop] to [value]. ("help properties"
                           for valid properties.) For properties taking list
                           values, [value] should be a comma-separated list of
                           values.

NAS:configuration storage config> done

After the readzilla has been added the pool looked like this - note the extra readzilla c7t0d0 in the cache section:

dwstor1:> confirm shell zpool status
pool: Oracle
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://www.sun.com/msg/ZFS-8000-9P
scan: resilvered 47.5K in 0h0m with 0 errors on Fri Feb 17 14:23:55 2012
config:

NAME STATE READ WRITE CKSUM
Oracle ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c5t5000C50019E42A14d0 ONLINE 0 0 0
c5t5000C50019FBE503d0 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
c5t5000C50019FBF8FFd0 ONLINE 0 0 0
c5t5000C50019FBF125d0 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
c5t5000C50019FC4CD4d0 ONLINE 0 0 0
c5t5000C50019FC5D41d0 ONLINE 0 0 5
mirror-3 ONLINE 0 0 0
c5t5000C50019FC509Bd0 ONLINE 0 0 0
c5t5000C50019FCE67Dd0 ONLINE 0 0 0
logs
mirror-4 ONLINE 0 0 0
c5tATASTECZEUSIOPS018GBYTESSTM0000D8A0Ad0 ONLINE 0 0 0
c5tATASTECZEUSIOPS018GBYTESSTM0000F5FD9d0 ONLINE 0 0 0
cache
c0t0d0 ONLINE 0 0 0
c0t1d0 ONLINE 0 0 0
c7t0d0 ONLINE 0 0 0
spares
c5t5000C50019FDA86Cd0 AVAIL
c5t5000C50019FD3114d0 AVAIL

errors: No known data errors

pool: system
state: ONLINE
scan: resilvered 4.80M in 0h0m with 0 errors on Fri Apr 15 11:21:41 2011
config:

NAME STATE READ WRITE CKSUM
system ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c1t1d0s0 ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0

errors: No known data errors

Removing Readzillas

There is no way for a customer to remove readzillas via the BUI or CLI. This requires a support call to be raised so an appropriately trained Oracle support analyst can carry out the procedure.

See https://stbeehive.oracle.com/teamcollab/wiki/AmberRoadSupport:Introduction+to+Readzillas for further details on removing Readzillas from a pool.

Check for relevancy - 10-May-2018

Attachments

This solution has no attachment