Asset ID:	1-77-1019006.1
Update Date:	2014-01-24
Keywords:

---

Solution Type  Sun Alert Sure

Solution  1019006.1 :   Collecting Support Data Using Common Array Manager while Drives are Bypassed May Cause Controllers to Reboot  


PreviouslyPublishedAs
231802

Bug Id
<BUG:15450082>

Product
Sun StorageTek 6130 Array
Sun StorageTek 6140 Array
Sun StorageTek 6540 Array
Sun StorageTek Flexline 240 Array
Sun StorageTek Flexline 280 Array
Sun StorageTek Flexline 380 Array
Sun StorageTek Common Array Manager Software 6.0

Date of Workaround Release
15-Feb-2008

Date of Resolved Release
04-Mar-2008

***Checked for relevance 24-Jan-2014***

1. Impact

Sun StorageTek 6130, 6140, and 6540 (with firmware 06.19.25.16) and Flexline 240, 280, and 380 arrays (with firmware 06.19.25.26) and collecting support data using Common Array Manager (CAM) 6.0 while drives are bypassed may cause controllers to reboot without warning. This may cause device path failovers or a loss of access due to both primary and secondary paths to a device being inaccessible.

2. Contributing Factors

This issue can occur on the following platforms:

• Sun StorEdge 6130 Array with firmware 06.19.25.16 or later
• Sun StorageTek 6140 Array with firmware 06.19.25.16 or later
• Sun StorageTek 6540 Array with firmware 06.19.25.16 or later
• Sun StorageTek Flexline 240 Array with firmware 06.19.25.26 or later
• Sun StorageTek Flexline 280 Array with firmware 06.19.25.26 or later
• Sun StorageTek Flexline 380 Array with firmware 06.19.25.26 or later

with:

• Sun StorageTek Common Array Manager 6.0

Notes:

1. For this issue to occur, one or more drives must be removed or in a "bypassed" state (removal of an otherwise optimal drive will place it into a state of Bypassed).
2. This issue can only happen during array support data collection using Common Array Manager(CAM) 6.0, specifically with the "Capture State" function of the collection which generates the stateCaptureData.dmp file.  This includes collection of data by both the CAM Service Advisor and supportData collection utilities.
   

3. Symptoms

At minimum, systems will experience device path fail-overs.  At worst, there will be a loss of access due to both primary and secondary paths to a device being inaccessible.

This issue can only occur during array support data collection (see Contributing Factors).  The system must also have at least one drive in a "Bypassed" state.  This condition can be reviewed by looking at the following:

Browser:

SSCS:

Host message events showing SCSI or Fibre Channel messages indicating loss of access to one or both array controllers are the most common symptom associated with this issue.  Hosts can also log path failovers as one or both controllers go through the boot cycle.

CAM may throw an error during collection, similar to the following:

Exception while accessing hids,108 on Controller A:
devmgr.versioned.jrpc.RPCError: TIMEOUT
Exception while accessing hids,108 on Controller B:
devmgr.versioned.jrpc.RPCError: TIMEOUT

This can be found during the collection using the Service Advisor, or by reviewing the "stateCaptureData.dmp" file included in the support data collection.

4. Workaround

This issue can be avoided by not collecting support data when you know that a drive has been removed or bypassed by the storage subsystem.  For this case, you can collect the data using the "service" command.

The following CLI commands can be run to obtain support data for CAM:

service -d <array_name> -c print -t arrayprofile > arrayProfileSummary.txt

service -d <array_name> -c print -t mel > majorEventLog.txt

service -d <array_name> -c read -q nvsram region=0xEE > NVSRAMdata.txt

service -d <array_name> -c print -t rls > readLinkStatus.csv

sscs list alarm > alarms.txt

5. Resolution

This issue is addressed in the following release:

• Common Array Manager 6.0.1 and later

for all affected arrays listed in Contributing Factors.


Modification History
04-Mar-2008: Updated Resolution section, RESOLVED
26-mar-2008: Updated Workaround section
10-Jul-2008: Updated Workaround for clarification
24-Jan-2014: Checked for currency/relevance/formatting; no change in content

Internal Comments
Pending release on the SDLC, CAM 6.0.1.11 will have the "hids 108"
command removed from the support data collection. Avoid running
'hids' altogether during serial sessions, especially when any drive is
in a non-optimal state.

Questions regarding this document should be addressed to
sunalertpublication_us_grp@oracle.com and copy the
responsible engineer/submitter listed below.

Internal Contributor/submitter
curtis.decotis@oracle.com
Internal Eng Responsible Engineer
stephen.brooks@oracle.com, brian.hanvey@oracle.com
Internal Services Knowledge Engineer
david.mariotto@oracle.com
Internal Eng Business Unit Group
NWS (Network Storage)
Internal Escalation ID
1-23211257

The excLogShow will show:



Stack Trace:

======== STACK SHOW ========

Showing for task id = 0xa280840 (symTask2), Running

FP=0xa280498, SP=0xa280154, PC=0x1348d50

Current executing task id = 0xa280840 (symTask2); not interrupted



Frame Ptr   Ret Addr  Return Name +
Offset             
Called Name + Offset

========== ========== ================================

========================

0x0a2807f0 0x00139bc8 vxTaskEntry +
0x2c               
vkiTask

0x0a28078c 0x0005d824 vkiTask +
0xd4                   
srcOpTask

0x0a280748 0x012f29c0 srcOpTask +
0x120                
cmdProcess

0x0a280730 0x014e5dbc cmdProcess + 0x74

symSYMbolCommandHandler

0x0a2806e0 0x016caa8c symSYMbolCommandHandler + 0x54   
svciov_dispatch

0x0a280630 0x016dc3b4 svciov_dispatch +
0x3ec          
stateCapture_1

0x0a280610 0x016c8ac8 stateCapture_1 +
0x38            
systemStateCapture

0x0a280530 0x0171e8c0 systemStateCapture +
0x5a0        hids

0x0a280510 0x01355f44 hids + 0x4c


0x0a2804e8 0x01356048 show__Q23hid10HidManagerUlUlUl + 0x90

showFunc__Q23hid10HidManagerP9HIDS_INFO

0x0a2804d0 0x013530e4 showFunc__Q23hid10HidManagerP9HIDS_INFO + 0x2ac

traverseSingleList__Q23hid10HidManagerP9HIDS_INFO


+ 0xe4  show__CQ23utl14ShowableObjecti

0x0a28048c 0x0182e138 show__CQ23utl14ShowableObjecti + 0xcc

toString__CQ23hid2LUi

           0x01348d50
toString__CQ23hid2LUi + 0x5a4

********

Task Id:         0xa280840

Name:           
"symTask2"

Status:          0x00
(ready)

Options:         0x0004
(dealloc_stk)

Priority:        125


Stack end:       0xa27cea4 (adjusted for
name)

Stack size:      0x399c (14748)

Stack margin:    0x1708 (5896)

Stack limit:     0xa27ceb0

Pend queue:      0x98c1b10

Last errno:      0x1c0001

References

Attachments

This solution has no attachment