Sun ZFS Storage Appliance: DE2-24C / DE2-24P Enclosures Are NOT Recognized By 7120 On 2011.1.5.0

Asset ID:	1-72-1539460.1
Update Date:	2013-05-17
Keywords:

Solution Type Problem Resolution Sure

Solution 1539460.1 : Sun ZFS Storage Appliance: DE2-24C / DE2-24P Enclosures Are NOT Recognized By 7120 On 2011.1.5.0

Applies to:

Sun ZFS Storage 7120 - Version All Versions to All Versions [Release All Releases]
7000 Appliance OS (Fishworks)
Regardless of statement in the release note on 2011.1.5.0 and the data sheet, Sun ZFS Storage 7120 does not recognize DE2-24P and DE2-24C enclosures with 2011.1.5.0 base version.

Symptoms

The following symptoms are observed. The examples will follow.

The disk enclosures are not visible from Configuration > Storage in the BUI/CLI.
The disk enclosures are not visible from Maintenance > Hardware in the BUI/CLI.
However the enclosures can be seen connected at the SAS level.

How to confirm your enclosure is not present at "configuration storage" via CLI.
By running the following commands, you are able to confirm the DE2 enclosures are not recognized in "configuration storage" context.

ar1:> configuration storage
ar1:configuration storage> config pool-0

Verify and allocate devices. Devices may be added on a per-device basis,
however SATA devices in SAS-1 enclosures may be added in half- or whole-chassis
units only. While affected devices may be allocated, they will not be available
for use and cannot be added later without reconfiguring the pool; for best
results, defer configuring storage until any problems can be repaired. Mixing
devices of differing speeds within a storage pool is strongly discouraged.

(.......snip.......)

ar1:configuration storage verify> show

        ID     STATUS   ALLOCATION   DATA          LOG           CACHE       RPM
        --    -------   ----------   -----------   -----------   ----------- -----
         0         ok       custom   11/11 30.0T   1/1   68.4G   0           7200 <==== internal drives only.
                                     -----------   -----------   -----------
                                     11    30.0T   1     68.4G   0

How to confirm your exclosure is not present via CLI.

By running the following commands, you can see if you have enclosure or not.
In this scenario, you only see 1 chassis (chassis-000) ONLY regardless of connected enclosure.

ar1:> maintenance hardware
ar1:maintenance hardware> list
NAME STATE MANUFACTURER MODEL SERIAL RPM
chassis-000 zfs01 ok Oracle Sun ZFS Storage 7120 1308FMM002 7200

How to confirm if your enclosure and hard drives are recognized at SAS level.

There is no supported way for end-users to confirm if the drives are recognized at the SAS level.

Please contact Oracle Support.

The following procedure describes how you can check pmcs devices using mdb. This procedure is for Oracle personnel only.

ar1# echo '::pmcs -T' | md -k
         Address   St Chip I B C   RG WorkFlgs SNo DMsk              DIP
==========================================================================
fffff60015d66000 Run 8001 0 0 0    0 00000000 44f6 0071 fffff600000e52e0　<==== Internal drives are either 11 or 12 drives. So this one is an entry for the internal HBA, installed on PCIe#3.
         Occupied PHYs: 14 (13 SAS + 0 SATA + 1 SMP)
    Configured targets: 14 (13 SAS + 0 SATA + 1 SMP)

         Address   St Chip I B C   RG WorkFlgs SNo DMsk              DIP
==========================================================================
fffff60019178000 Run 8001 1 0 0    0 00000000 403e 0071 fffff600000e40a8 <==== External drives are 24 drives, dual path

         Occupied PHYs: 52 (50 SAS + 0 SATA + 2 SMP)
    Configured targets: 52 (50 SAS + 0 SATA + 2 SMP)

7120 has either 11 or 12 drives on its chassis. The HBA with an internal SAS should Occupied PHYs of either 13 or 14. (Number of drives + 2, 1 for expander and 1 for SMP address.)
This HBA is installed on PCIe#3 slot.

The external HBA on 7120 is on PCIe#4 slot. Since DE2-24 has 20 data drives and 0 to 4 SSDs regardless of height. After adding 2 PHYs for each expander or IOM, this will give us either 44,46,48,50 or 52.

The table below is the quick reference.

Number of SSDs	Number of total Drives	Number of SAS Addresses (Drives)	Number of SAS Addresses (IOM)	Number of SMP Addresses	Total
0	20	40	2	2	44
1	21	42	2	2	46
2	22	44	2	2	48
3	23	46	2	2	50
4	24	48	2	2	52

The command below allows you to list up the targets under the HBA. Notice that I am quoting the address of HBA instance from pmcs -T output.

ar1# echo 'fffff60019178000::pmcs -p' | mdb -k
         Address   St Chip I B C   RG WorkFlgs SNo DMsk              DIP
==========================================================================
fffff60019178000 Run 8001 1 0 0    0 00000000 4022 0071 fffff600000e40a8

       PHY information
       --------
       Level 0
       --------
       SAS Address         Hdl Phy# Speed Type
       508002000144d23f 0x0000    3 6 Gb/s EXP Path: pp03
          --------
          Level 1
          --------
          SAS Address         Hdl Phy# Speed Type
          5000cca03e2b0a59 0x0001    0 6 Gb/s SAS Path: pp03.00
          5000cca03e2404c5 0x0002    1 6 Gb/s SAS Path: pp03.01
          5000cca03e2b0795 0x0003    2 6 Gb/s SAS Path: pp03.02
          5000cca03e2b4a11 0x0004    3 6 Gb/s SAS Path: pp03.03
          5000cca03e2937b9 0x0005    4 6 Gb/s SAS Path: pp03.04
          5000cca03e27a811 0x0006    5 6 Gb/s SAS Path: pp03.05
          5000cca03e2d0aed 0x0007    6 6 Gb/s SAS Path: pp03.06
          5000cca03e2c902d 0x0008    7 6 Gb/s SAS Path: pp03.07
          5000cca03e2452d1 0x0009    8 6 Gb/s SAS Path: pp03.08
          5000cca03e27a785 0x000a    9 6 Gb/s SAS Path: pp03.09
          5000cca03e2a06a1 0x000b   10 6 Gb/s SAS Path: pp03.0a
          5000cca03e2a095d 0x000c   11 6 Gb/s SAS Path: pp03.0b
          5000cca03e26c939 0x000d   12 6 Gb/s SAS Path: pp03.0c
          5000cca03e2404dd 0x000e   13 6 Gb/s SAS Path: pp03.0d
          5000cca03e26c4f5 0x000f   14 6 Gb/s SAS Path: pp03.0e
          5000cca03e2b09d5 0x0010   15 6 Gb/s SAS Path: pp03.0f
          5000cca03e2a09cd 0x0011   16 6 Gb/s SAS Path: pp03.10
          5000cca03e23d8b5 0x0012   17 6 Gb/s SAS Path: pp03.11
          5000cca03e240ac1 0x0013   18 6 Gb/s SAS Path: pp03.12
          5000cca03e2a0985 0x0014   19 6 Gb/s SAS Path: pp03.13
          5000cca03e2b0601 0x0015   20 6 Gb/s SAS Path: pp03.14
          5000cca03e2b2391 0x0016   21 6 Gb/s SAS Path: pp03.15
          5000cca03e2d0b25 0x0017   22 6 Gb/s SAS Path: pp03.16
          5000cca03e2a099d 0x0018   23 6 Gb/s SAS Path: pp03.17
          508002000144d23e 0x0019   36 6 Gb/s SAS Path: pp03.24

       508002000144c23f 0x001a    7 6 Gb/s EXP Path: pp07
          --------
          Level 1
          --------
          SAS Address         Hdl Phy# Speed Type
          5000cca03e2b0a5a 0x001b    0 6 Gb/s SAS Path: pp07.00
          5000cca03e2404c6 0x001c    1 6 Gb/s SAS Path: pp07.01
          5000cca03e2b0796 0x001d    2 6 Gb/s SAS Path: pp07.02
          5000cca03e2b4a12 0x001e    3 6 Gb/s SAS Path: pp07.03
          5000cca03e2937ba 0x001f    4 6 Gb/s SAS Path: pp07.04
          5000cca03e27a812 0x0020    5 6 Gb/s SAS Path: pp07.05
          5000cca03e2d0aee 0x0021    6 6 Gb/s SAS Path: pp07.06
          5000cca03e2c902e 0x0022    7 6 Gb/s SAS Path: pp07.07
          5000cca03e2452d2 0x0023    8 6 Gb/s SAS Path: pp07.08
          5000cca03e27a786 0x0024    9 6 Gb/s SAS Path: pp07.09
          5000cca03e2a06a2 0x0025   10 6 Gb/s SAS Path: pp07.0a
          5000cca03e2a095e 0x0026   11 6 Gb/s SAS Path: pp07.0b
          5000cca03e26c93a 0x0027   12 6 Gb/s SAS Path: pp07.0c
          5000cca03e2404de 0x0028   13 6 Gb/s SAS Path: pp07.0d
          5000cca03e26c4f6 0x0029   14 6 Gb/s SAS Path: pp07.0e
          5000cca03e2b09d6 0x002a   15 6 Gb/s SAS Path: pp07.0f
          5000cca03e2a09ce 0x002b   16 6 Gb/s SAS Path: pp07.10
          5000cca03e23d8b6 0x002c   17 6 Gb/s SAS Path: pp07.11
          5000cca03e240ac2 0x002d   18 6 Gb/s SAS Path: pp07.12
          5000cca03e2a0986 0x002e   19 6 Gb/s SAS Path: pp07.13
          5000cca03e2b0602 0x002f   20 6 Gb/s SAS Path: pp07.14
          5000cca03e2b2392 0x0030   21 6 Gb/s SAS Path: pp07.15
          5000cca03e2d0b26 0x0031   22 6 Gb/s SAS Path: pp07.16
          5000cca03e2a099e 0x0032   23 6 Gb/s SAS Path: pp07.17
          508002000144c23e 0x0033   36 6 Gb/s SAS Path: pp07.24
         Occupied PHYs: 52 (50 SAS + 0 SATA + 2 SMP)

Changes

New DE2-24 tray has been installed and the installed code is 2011.1.5 (non IDR release).

Cause

2011.1.5.0 for Iwashi+ package does not have necessary configuration modules for recognizing DE2-24 enclosures.
(This is for base release version of 2011.1.5.0. It may change in future.)

Solution

Upgrade to 2011.1.6.0.

Please contact Oracle Support for resolution if you have the reason that you cannot update from 2011.1.5.0.

The status as of March 26, 2013.

We have an IDR ak-nas-2011-04-24-5-0-1-2-33-16-1.pkg.gz.

You need an RPE approval to release this IDR to a customer.

If you are an L1 engineer, open a collab SR and ask him/her to provide you this IDR.

2011.1.6.0 has the code for 7120 to recognize the enclosures.

References

<BUG:16512569> - DE2-24C DETECTED BY 7120 AT THE HARDWARE LEVEL, BUT AKD DOES NOT SEE IT

Attachments

This solution has no attachment