Common Array Manager (CAM) Reports a Health Status of "No Contact" on a Sun Storage 2500, 2500-M2 or J4000 Array

Asset ID:	1-72-1607209.1
Update Date:	2015-06-26
Keywords:

Solution Type Problem Resolution Sure

Solution 1607209.1 : Common Array Manager (CAM) Reports a Health Status of "No Contact" on a Sun Storage 2500, 2500-M2 or J4000 Array

Applies to:

Sun Storage 2530 Array - Version Not Applicable and later
Sun Storage J4200 Array - Version All Versions and later
Sun Storage J4400 Array - Version All Versions and later
Sun Storage J4500 Array - Version All Versions and later
Sun Storage 2510 Array - Version Not Applicable and later
Information in this document applies to any platform.

Symptoms

The Storage System Summary View in CAM reports the health of a managed array as "No Contact". Similarly, CAM CLI (sscs) reports the same.

# sscs list array J4200_1
Array: J4200_1
Array:             J4200_1
Array Type:        j4200
Firmware Version: 3A32
Health Status:     No Contact
Id:                20:29:QT:F0:80:9Q:CJ:01:4
Network Address:   10.152.224.202
Serial Number:     SUN.j4200.2029QTF0809QCJ014
Total Capacity:    3,600,000,170,124 Bytes

The sscs CAM command-line interface is found in the following locations:

Solaris: /opt/SUNWstkcam/bin/
Linux: /opt/sun/cam/bin/
Windows: C:\Program Files\Sun\Common Array Manager\bin

When the array is selected, all of the reporting and inquiry information appears to be working. All FRU's show up with a respective status, revision, and health.

The secondary symptom shows up when any type of firmware upgrade is attempted. The upgrade will fail regardless of the technique used.

Install Firmware Baseline from the Storage System Summary View in CAM
Correcting an IOM or SIM Firmware Mismatch from Service Advisor
Upgrade Firmware from Service Advisor
Using cli
      # /opt/SUNWstkcam/bin/sscs modify -a J4200_1 -o -t sim firmware
      WARNING: This command will load new firmware if needed and may impact
      array management and data availability.
      Do you wish to continue? [y,n] : y
      Firmware install failed.

The csmservice utility will also fail in a similar fashion.
# /opt/SUNWsefms/bin/csmservice -i -a J4200_1 -o
sa.cmd.error
Firmware install failed.

Changes

All of these problems typically originate during firmware upgrades. The upgrade may be an overall, array upgrade during planned downtime, or an upgrade of an individual component which was just replaced.

Cause

CAM uses a locking mechanism to prevent any other management activity during an upgrade. Once the upgrade completes, the lock is cleared. Even when the upgrade fails, the exit scripts provide for removal of these locks. This problem will occur if, for some reason, the lock remains after the firmware upgrade.

Solution

The fix is to simply remove the lock file and re-establish the management connection to the array. You can find the lock file in..

Solaris /var/opt/SUNWsefms/MgmtLocks
Windows \Program Files (x86)\Sun\Common Array Manager\Component\fms\var\MgmtLocks
Linux /var/opt/sun/cam/fms/MgmtLocks

First, you remove the lock file.

# cd /var/opt/SUNWsefms/MgmtLocks
# ls
MASTERLOCK SUN.j4200.2029QTF0809QCJ014.lock
# rm SUN.j4200.2029QTF0809QCJ014.lock
# ls
MASTERLOCK

Followed by a refresh of Fault Management Services. (Many ways to do this. In this example, the Solaris svcadm command is used)

# svcs -a | grep fms
    online         11:11:06 svc:/system/fmservice:default
# svcadm restart svc:/system/fmservice:default
    Wait 30 seconds.
# svcs -a | grep fms
    online         11:12:45 svc:/system/fmservice:default

Solaris 8,9: /opt/SUNWsefms/sbin/fmservice.sh restart
Linux : /opt/sun/cam/private/fms/sbin/fmservice.sh restart
Windows : Use control panel to restart Sun_STK_FMS

Final step will be to remove and re-register the array in CAM (Once again, many ways to do this.)

# sscs remove -a J4200_1 registeredarray

# sscs add -i 127.0.0.1 registeredarray
Name        Type Network Address          Serial Number
----------- ----- ------------------------ ---------------------------
J4200_1     j4200 10.152.224.202 (In-band) SUN.j4200.2029QTF0809QCJ014

# sscs list array J4200_1
Array: J4200_1
Array:             J4200_1
    Array Type:        j4200
    Firmware Version: 3A32
Health Status:     OK
    Id:                20:29:QT:F0:80:9Q:CJ:01:4
Network Address:   10.152.224.202
Serial Number:     SUN.j4200.2029QTF0809QCJ014
    Total Capacity:    3,600,000,170,124 Bytes

Attachments

This solution has no attachment