Pillar MaxRep: Replication Plan Stuck on "Configuring LUN Protection" Status

Asset ID:	1-72-1528789.1
Update Date:	2016-06-21
Keywords:

Solution Type Problem Resolution Sure

Solution 1528789.1 : Pillar MaxRep: Replication Plan Stuck on "Configuring LUN Protection" Status

Applies to:

Pillar Axiom Replication Engine (MaxRep) - Version 2.0 to 2.0 [Release 2.0]
Information in this document applies to any platform.

Symptoms

After SAN related issues, a protection plan may not re-sync when a manual re-sync is forced. The plan stays in "Configuring LUN Protection" status.

To discuss this information further with Oracle experts and industry peers, we encourage you to review, join or start a discussion in the My Oracle Support Community - Disk Storage Pillar Axiom System

Changes

SAN issues interrupted the replication process.

Cause

The following events can be observed in the source or destination AXIOM events: SAN_EVENT_INM_MIRROR_FAILED this is typical of SAN issues where the SAN connection cannot be maintained.

Solution

First an investigation of the SAN is recommended. In similar cases, the SAN was undersized particularely ISLs between the source and destination axiom leading to bottlenecks high latency and frames dropped.

Secondly to recover the plan:

1. Note which LUNs are source and destination

2. Delete or force delete the protection plan.

3. Go in Manage protection plan, Delete.

4. Tick the "Clean CDP Retention logs" checkbox.

5. Click delete

6. Once the protection plan has been deleted, recreate it and synchronise the plan.

This issue is being addressed to engineering and a future release will add improvements to avoid deletion when such a case happens.

Other cause of Replication Pair stuck on “Configuring LUN Protection”

The following issue has the same symptom as the above (seen under SR 3-9764016301 / Bug 19889416)

Fact: Force deleting the protection plans without clicking the “Clean CDP Retention logs” will create stale entries in db file.

Analysis:

After analyzing the logs we found some stale entries in database as part of force deleting earlier protection plan which is causing the issue.

From axiom_pcli.log:

2014-10-31 15:50:10 EXCEPTION : ARRAY_PCLI : getMirrorLunStatus : failed with error : output : Message

Response

CorrelationID: 1414767010

BeginStreamResponse

TaskGuid: 4130303437303742A13F1EE81852D74E

TaskFqn: /GetMirrorLunStatus/2696992/replication

Message

Response

CorrelationID: 1414767010

ErrorList

ErrorInformation[0]

ErrorCode: LUN_DOES_NOT_EXIST

ParameterErrorInformation[0]

InvalidParameterStringId: /Message/Request/GetMirrorLunStatus/Identity[1]/Fqn

ErrorCode: IDENTIFIER_NOT_FOUND

EndStreamResponse

TaskGuid: 4130303437303742A13F1EE81852D74E

TaskFqn: /GetMirrorLunStatus/2696992/replication

From svagent.log, we found the pair in state "41":

(11-04-2014 12:20:56): DEBUG 1616 47875974269104 dataprotection is not running on deviceName = /dev/mapper/2000b080001004708

(11-04-2014 12:20:56): DEBUG 1616 47875974269104 EXITED PerformPendingPauseAction

(11-04-2014 12:20:56): DEBUG 1616 47875974269104 volume /dev/mapper/2000b080001004708 state (41) is not in-progress.

Cause: This is a known issue with MaxRep R2, the customer must always select the option “Clean CDP Retention Logs” when doing a delete.

Workaround:

Deactivate all the protection plans.
Force delete the replication pairs which are stuck in deletion pending.
Delete all stale entries from database manually (raise a Bug with MaxRep Sustaining Engineering and schedule a WebEx with the customer and Engineering)
Activate all the Protection Plans.
Check if the replication pairs are progressing successfully after performing the above steps.

Note: You need to involve Engineering team for the above activity.

Fix: A patch to fix this is still under development phase.

Attachments

This solution has no attachment