MaxRep: Deactivating or Deleting a Protection Plan with Queued Pairs Can Lead to Pairs Stuck in Configure LUN Protection State

Asset ID:	1-72-2006243.1
Update Date:	2018-04-25
Keywords:

Solution Type Problem Resolution Sure

Solution 2006243.1 : MaxRep: Deactivating or Deleting a Protection Plan with Queued Pairs Can Lead to Pairs Stuck in Configure LUN Protection State

Applies to:

Pillar Axiom Replication Engine (MaxRep) - Version 2.0 to 3.0 [Release 2.0 to 3.0]
Information in this document applies to any platform.

Symptoms

One or multiple Pairs within the same Protection Plan are stuck in Configure LUN Protection state following the activation of the Protection Plan.

Changes

The user previously deactivated or deleted the Protection Plan while some of the LUNs were in a Queued state and decided to reuse the same LUNs in the pairs by activating the Protection Plan again.

Cause

This issue is happening when the Batch Resync feature is enabled. It can be verified on the MaxRep GUI under the “Protect -> Manage Protection Plan” page.
Then for each Protection Plan go to “Modify -> Modify Replication Options”.

Batch Resync feature

In this example, we allow 2 pairs to resync at a time: if the Protection Plan has more than 2 pairs, only the first 2 will be resync'd simultaneously while the others will stay in the Queued state. Once one of the two pairs transitions to Diff Sync, the next pair in the queue will go into Resync.

The MaxRep software might get stale entries in its database if the user decides to deactivate or delete the Protection Plan while at least one pair is in a Queued state. The user might experience the above symptom when the same LUNs that were in a Queued state are synchronized (within the same Protection Plan or a new Protection Plan).

Example of a situation where the Protection Plan should not be deactivated or deleted:

Queued Pairs

Solution

This situation requires the assistance of Oracle Support. Please open a Service Request and refer to this document.

Deactivate the protection plan that has pair(s) stuck in Configure LUN Protection state.
Force-delete the replication pairs if they are stuck in deletion pending state for long time (30-60 minutes).
At this stage we will have stale entries in MaxRep and in DB.

SSH to the MaxRep Engine
Connect to the CS database:

For MaxRep R2:

[root@MAXREP ~]# mysql svsdb1

For MaxRep R3:

[root@MAXREP ~]# mysql -u root -psvsHillview svsdb1

Collect the stale AT LUN Mapping Ids from CS database using the following query:

mysql> SELECT DISTINCT applianceTargetLunMappingId FROM hostApplianceTargetLunMapping WHERE sharedDeviceId NOT IN (SELECT Phy_Lunid FROM srcLogicalVolumeDestLogicalVolume);

In this example, the command returns:

+-----------------------------+
| applianceTargetLunMappingId |
+-----------------------------+
| inmage0000000041 |
+-----------------------------+
1 row in set (0.00 sec)

mysql>

The command provides the list of AT LUNs with a stale entry that were created in MaxRep and exported to the Axiom/FS1. We need to un-export and delete these AT LUN(s).
Use the following command to un-export the AT LUNs (run it for each Axiom PWWN):

[root@MAXREP ~]# echo “del <AT LUN Name>” > /proc/scsi_tgt/groups/<Axiom PWWN>/devices

Example:

[root@MAXREP ~]# echo “del inmage0000000041” > /proc/scsi_tgt/groups/21:00:00:0b:08:04:1f:10/devices
[root@MAXREP ~]# echo “del inmage0000000041” > /proc/scsi_tgt/groups/22:00:00:0b:08:04:1f:10/devices
[root@MAXREP ~]# echo “del inmage0000000041” > /proc/scsi_tgt/groups/23:00:00:0b:08:04:1f:10/devices
[root@MAXREP ~]# echo “del inmage0000000041” > /proc/scsi_tgt/groups/24:00:00:0b:08:04:1f:10/devices

Then run this command to delete the AT LUN:

[root@MAXREP ~]# /usr/local/InMage/Vx/bin/inm_dmit --op=del_lun --lun_name=<AT Lun Name>

Example:

[root@MAXREP ~]# /usr/local/InMage/Vx/bin/inm_dmit --op=del_lun --lun_name=inmage0000000041
Fabric lun inmage0000000041 deleted successfully

Reconnect to the CS database and run the following query to clean the stale entry from the CS database:

mysql> DELETE FROM sharedDevices WHERE sharedDeviceId NOT IN (SELECT Phy_Lunid FROM srcLogicalVolumeDestLogicalVolume);

Example:

mysql> DELETE FROM sharedDevices WHERE sharedDeviceId NOT IN (SELECT Phy_Lunid FROM srcLogicalVolumeDestLogicalVolume);
Query OK, 1 row affected (0.04 sec)

Re-run the 1st query to make sure the stale entry is no longer there:

mysql> SELECT DISTINCT applianceTargetLunMappingId FROM hostApplianceTargetLunMapping WHERE sharedDeviceId NOT IN (SELECT Phy_Lunid FROM srcLogicalVolumeDestLogicalVolume);
Empty set (0.00 sec)

References

<BUG:19889416> - AFTER MAXREP ENGINE FAILURE, "TDATABASE" SAN PROTECTION PLAN IS STUCK IN "CONFIG

Attachments

This solution has no attachment