Oracle ZFS Storage Appliance: How to Configure and Present LUNs Over Fiber Channel to a Solaris Client

Asset ID:	1-71-1945992.1
Update Date:	2018-05-04
Keywords:

Solution Type Technical Instruction Sure

Solution 1945992.1 : Oracle ZFS Storage Appliance: How to Configure and Present LUNs Over Fiber Channel to a Solaris Client

Applies to:

Sun Storage 7410 Unified Storage System - Version All Versions and later
Sun Storage 7210 Unified Storage System - Version All Versions and later
Sun Storage 7310 Unified Storage System - Version All Versions and later
Oracle ZFS Storage ZS3-2 - Version All Versions and later
Oracle ZFS Storage ZS3-4 - Version All Versions and later
7000 Appliance OS (Fishworks)

Goal

The document provides a working example of presenting a LUN from an Oracle ZFS Appliance to a Solaris client over fibre channel.

The connection between the two is established through a fabric switch.

The commands used and outputs generated provide a guide line for creating and troubleshooting similar configurations.

Solution

Start off by finding the World Wide Port Numbers (wwpn) of the HBA on the Solaris client.

# fcinfo hba-port

HBA Port WWN: 210000e08b92d894
        OS Device Name: /dev/cfg/c2
        Manufacturer: QLogic Corp.
        Model: 375-3294-01
        Firmware Version: 05.03.02
        Type: N-port
        State: online
        Current Speed: 4Gb
        Node WWN: 200000e08b92d894
HBA Port WWN: 210100e08bb2d894
        OS Device Name: /dev/cfg/c3
        Manufacturer: QLogic Corp.
        Model: 375-3294-01
        Firmware Version: 05.03.02
        Type: N-port
        State: online
        Current Speed: 4Gb
        Node WWN: 200100e08bb2d894

The wwpn from each HBA will be defined as an initiator on the appliance.

NAS-7000-b:> configuration san fc initiators create
NAS-7000-b:> configuration san fc initiators initiator-000 (uncommitted)> set alias=E280R-Port1
                     alias = E280R-Port1 (uncommitted)
NAS-7000-b:> configuration san fc initiators initiator-000 (uncommitted)> set initiator=wwn.210100E08BB2D894
                     initiator = wwn.210100E08BB2D894 (uncommitted)
NAS-7000-b:> configuration san fc initiators initiator-000 (uncommitted)> commit

NAS-7000-b:> configuration san fc initiators create
NAS-7000-b:> configuration san fc initiators initiator-001 (uncommitted)> set alias=E280R-Port2
                         alias = E280R-Port2 (uncommitted)
NAS-7000-b:> configuration san fc initiators initiator-001 (uncommitted)> set initiator=210000e08b92d894
                     initiator = wwn.210000E08B92D894 (uncommitted)
NAS-7000-b:> configuration san fc initiators initiator-001 (uncommitted)> commit

NAS-7000-b:> configuration san fc initiators list
                    NAME          ALIAS
                    initiator-000 E280R-Port1
                                  |
                                  +-> INITIATOR
                                      wwn.210100E08BB2D894
                    initiator-001 E280R-Port2
                                  |
                                  +-> INITIATOR
                                      wwn.210000E08B92D894

You can now put these initiators into an initiator group. You will use the initiator group to exclusively present LUNs to this specific client.

Similarly, you will configure the fibre channel HBA's on the appliance as targets. The targets will then be put into a target group.

In this example, Port 1 is used from each HBA in each cluster head.

NAS-7000-a:> configuration san fc targets list
   NAME       MODE       WWN                     PORT                 SPEED
   target-000 target     wwn.2100001B328E0682    PCIe 0: Port 1       4 Gbit/s
   target-001 target     wwn.2101001B32AE0682    PCIe 0: Port 2       4 Gbit/s

NAS-7000-b:> configuration san fc targets list
   NAME       MODE       WWN                     PORT                 SPEED
   target-000 target     wwn.21000024FF3E0B68    PCIe 0: Port 1       4 Gbit/s
   target-001 target     wwn.21000024FF3E0B69    PCIe 0: Port 2       4 Gbit/s

NAS-7000-a:> configuration san fc targets groups create
NAS-7000-a:> configuration san fc targets group-000 (uncommitted)> set name=SAN1-pcie0port1
                       name = SAN1-pcie0port1 (uncommitted)
NAS-7000-a:> configuration san fc targets group-000 (uncommitted)> set targets=wwn.2100001B328E0682,wwn.21000024FF3E0B69
                       targets = wwn.2100001B328E0682,wwn.21000024FF3E0B69 (uncommitted)
NAS-7000-a:> configuration san fc targets group-000 (uncommitted)> commit

NAS-7000-a:> configuration san fc targets groups select group-000 show
                 Properties:
                   name = SAN1-pcie0port1
                   targets = wwn.2100001B328E0682,wwn.21000024FF3E0B69

If the fibre channel card is set up as an Initiator, you need to reconfigure it into a target before proceeding.
NAS-7000-a:> configuration san fc targets list
   NAME       MODE       WWN                     PORT                 SPEED
   target-000 initiator wwn.2100001B328E0682    PCIe 0: Port 1       4 Gbit/s
   target-001 initiator wwn.2101001B32AE0682    PCIe 0: Port 2       4 Gbit/s

NAS-7000-a:> configuration san fc targets set targets=wwn.2101001B32AE0682,wwn.2100001B328E0682
(Reboot required to toggle the FC HBA from an initiator to a target)

Two LUNs are created. Each is bound to the initiator group and target group previously defined.

This guarantees that the LUN will only be presented to the Solaris client defined.

NAS-7000-a:> shares select FC-Lun lun d-base
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-base (uncommitted)> set volsize=50G
               volsize = 50G (uncommitted)
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-base (uncommitted)> set targetgroup=SAN1-pcie0port1
               targetgroup = SAN1-pcie0port1 (uncommitted)
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-base (uncommitted)> set initiatorgroup=E280R-Server
               initiatorgroup = E280R-Server (uncommitted)
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-base (uncommitted)> commit

NAS-7000-a:> shares select FC-Lun lun d-logs
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-logs (uncommitted)> set volsize=2G
               volsize = 2G (uncommitted)
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-logs (uncommitted)> set targetgroup=SAN1-pcie0port1
               targetgroup = SAN1-pcie0port1 (uncommitted)
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-logs (uncommitted)> set initiatorgroup=E280R-Server
               initiatorgroup = E280R-Server (uncommitted)
NAS-7000-a:> shares (pool-de24p) FC-Lun/d-logs (uncommitted)> commit

Use the shell command stmfadm on the appliance to see the LUNs created

NAS-7000-a# stmfadm list-lu
LU Name: 600144F0D2017CFB0000546251350001
LU Name: 600144F0D2017CFB0000546251860002
NAS-7000-a# stmfadm list-lu -v 600144F0D2017CFB0000546251350001
LU Name: 600144F0D2017CFB0000546251350001
    Operational Status     : Online
    Provider Name          : sbd
    Alias                  :
    View Entry Count       : 1
    Data File              : /dev/zvol/rdsk/pool-de24p/local/FC-Lun/d-base
    Meta File              : not set
    Size                   : 53687091200
    Block Size             : 512
    Management URL         : https://10.145.229.132:215/ak https://192.168.10.132:215/ak
    Software ID            : 000001
    Vendor ID              : SUN
    Product ID             : ZFS Storage 7320
    Serial Num             : not set
    Write Protect          : Disabled
    Write Cache Mode Select: Enabled
    Writeback Cache        : Disabled
    Access State           : Active

On the solaris client, start with luxadm to verify connectivity. In this example, you can see the target and initiator groups defined above are CONNECTED

# luxadm -e port
   /devices/pci@8,600000/SUNW,qlc@1/fp@0,0:devctl              CONNECTED
   /devices/pci@8,600000/SUNW,qlc@1,1/fp@0,0:devctl            CONNECTED

# luxadm -e dump_map /devices/pci@8,600000/SUNW,qlc@1/fp@0,0:devctl
   Pos Port_ID Hard_Addr Port WWN         Node WWN         Type
   0    10000   0         2100001b328e0682 2000001b328e0682 0x0 (Disk device)
   6    10100   0         210000e08b92d894 200000e08b92d894 0x1f (Unknown Type,Host Bus Adapter)

# luxadm -e dump_map /devices/pci@8,600000/SUNW,qlc@1,1/fp@0,0:devctl
   Pos Port_ID Hard_Addr Port WWN         Node WWN         Type
   2    20500   0         21000024ff3e0b69 20000024ff3e0b69 0x0 (Disk device)
   5    20100   0         210100e08bb2d894 200100e08bb2d894 0x1f (Unknown Type,Host

The format utility has discovered the LUNs.

# echo | format
AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <SUN72G cyl 14087 alt 2 hd 24 sec 424>
          /pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w21000004cf2cb0c7,0
       5. c5t600144F0D2017CFB0000546251350001d0 <SUN-ZFSStorage7320-1.0 cyl 1623 alt 2 hd 254 sec 254>
          /scsi_vhci/ssd@g600144f0d2017cfb0000546251350001
       6. c5t600144F0D2017CFB0000546251860002d0 <SUN-ZFSStorage7320-1.0 cyl 16382 alt 2 hd 8 sec 32>
          /scsi_vhci/ssd@g600144f0d2017cfb0000546251860002

Drill down and view the unique data paths associated with the virtual disk.

# luxadm display /dev/rdsk/c5t600144F0D2017CFB0000546251860002d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c5t600144F0D2017CFB0000546251860002d0s2
Vendor:               SUN
Product ID:           ZFS Storage 7320
Revision:             1.0
Serial Num:
Unformatted capacity: 2048.000 MBytes
Read Cache:           Enabled
    Minimum prefetch:   0x0
    Maximum prefetch:   0x0
Device Type:          Disk device
Path(s):

/dev/rdsk/c5t600144F0D2017CFB0000546251860002d0s2
/devices/scsi_vhci/ssd@g600144f0d2017cfb0000546251860002:c,raw
   Controller           /devices/pci@8,600000/SUNW,qlc@1/fp@0,0
    Device Address              2100001b328e0682,2
    Host controller port WWN    210000e08b92d894
    Class                       primary
    State                       ONLINE
   Controller           /devices/pci@8,600000/SUNW,qlc@1,1/fp@0,0
    Device Address              21000024ff3e0b69,2
    Host controller port WWN    210100e08bb2d894
    Class                       secondary
    State                       STANDBY
Note: For ALUA supported Operating Systems (such as Solaris 11, the secondary path will also report as ONLINE)

Similarly, mpathadm may be used in lieu of luxadm.

# mpathadm list lu
        /dev/rdsk/c5t600144F0D2017CFB0000546251860002d0s2
                Total Path Count: 2
                Operational Path Count: 2

# mpathadm show lu /dev/rdsk/c5t600144F0D2017CFB0000546251860002d0s2
Logical Unit: /dev/rdsk/c5t600144F0D2017CFB0000546251860002d0s2
        mpath-support: libmpscsi_vhci.so
        Vendor: SUN
        Product: ZFS Storage 7320
        Revision: 1.0
        Name Type: unknown type
        Name: 600144f0d2017cfb0000546251860002
        Asymmetric: yes
        Current Load Balance: round-robin
        Logical Unit Group ID: NA
        Auto Failback: on
        Auto Probing: NA

        Paths:
                Initiator Port Name: 210100e08bb2d894
                Target Port Name: 21000024ff3e0b69
                Override Path: NA
                Path State: OK
                Disabled: no

                Initiator Port Name: 210000e08b92d894
                Target Port Name: 2100001b328e0682
                Override Path: NA
                Path State: OK
                Disabled: no

        Target Port Groups:
                ID: 1
                Explicit Failover: no
                Access State: standby
                Target Ports:
                        Name: 21000024ff3e0b69
                        Relative ID: 256

                ID: 0
                Explicit Failover: no
                Access State: active
                Target Ports:
                        Name: 2100001b328e0682
                        Relative ID: 1

*** Check for relevancy - 04-May-2018 ***

Attachments

This solution has no attachment