Sun Storage 2500-M2 Arrays: Controller Replacement with Previously Used Controller May Cause Loss of I/O and Data Corruption

Asset ID:	1-72-1547607.1
Update Date:	2018-01-08
Keywords:

Solution Type Problem Resolution Sure

Solution 1547607.1 : Sun Storage 2500-M2 Arrays: Controller Replacement with Previously Used Controller May Cause Loss of I/O and Data Corruption

Applies to:

Sun Storage 2540-M2 Array - Version Not Applicable and later
Sun Storage 2530-M2 Array - Version Not Applicable and later
Information in this document applies to any platform.

Symptoms

An existing 2500-M2 array is running 07.77.xx.xx or 07.80.xx.xx code. An alarm is generated indicating a controller fault has occurred, with an appropriate action item for a controller replacement. If the replacement controller being inserted:

Contains 07.84.44.10 firmware
And has been previously used in another 2500-M2

Then, the replacement will fail, rendering the controller unusable. Typically:

The replacement controller will go into a boot loop.
It will reboot 5 times in a row and then lock down.
CAM will report the controller as offline.
The seven segment display will display 0S.

2500-M2 arrays already running 07.84.xx.xx firmware are not affected

Changes

The only change to the system was the current action item of replacing the faulted controller.

Cause

The problem is caused by the PSTOR, or Persistent Store/Storage on the controller being inserted. Because the replacement controller contains 07.84.xx.xx firmware, and was used in another array, it contains a PSTOR which is not understood by the 07.80.xx.xx controller. When this information is read, it results in corruption.

PSTOR manages a small region on each cache backup device and stores records pertaining to the cache backup devices.

The following messages may be observed in the controller shell :

03/25/13-13:26:03 (PersistentRestore): NOTE: PSTOR: PstorRecordManager::readRecord data block not found

03/25/13-13:26:03 (PersistentRestore): WARN: IOManager::readBackupStatus - Pstor record does not exsit
03/25/13-13:26:03 (PersistentRestore): NOTE: IOManager::getBackupDataSize - read to pstor failed

03/25/13-13:26:03 (PersistentRestore): WARN: ddcDq & ddcTrace restore abandoned: nothing to recover
03/25/13-13:26:03 (PersistentRestore): WARN: PSTOR: PstorRecordMgr: removeRecord failed
03/25/13-13:26:03 (PersistentRestore): WARN: IOManager::readBackupStatus - Pstor record does not exist
03/25/13-13:26:03 (PersistentRestore): NOTE: DDC Restore Failed

Solution

If your array fits the signature of this problem, contact Oracle support for assistance in returning the controller to optimal and recovering data. To recover the corruption the array will need to be reset to factory default and data recovered via a qualified recovery technique.

This problem will NOT occur if the replacment controller is one shipped directly from Oracle Logistics, or an on-site spare which has never been inserted into a 2500-M2.

Do not discuss the NetApp defect with customers.

PSTOR corruption is documented in Oracle Bug 16759315 which is associated with NetApp Defect LSIP200280245. If the corruption has already occurred, the array must be reset to a factory default and data recovered via a qualified recovery technique. Examples include use of the service recover, or restore from backup.

References

<BUG:16759315> - PSTOR CORRUPTED AFTER MOVING A 07.84 CONTROLLER INTO A 07.80 ARRAY

Attachments

This solution has no attachment