Solaris Cluster 3.x How to Change SCSI JBOD Disk Under Veritas Volume Manager 3.x

Asset ID:	1-71-1001992.1
Update Date:	2018-01-16
Keywords:

Solution Type Technical Instruction Sure

Solution 1001992.1 : Solaris Cluster 3.x How to Change SCSI JBOD Disk Under Veritas Volume Manager 3.x

Applies to:

Solaris Cluster - Version 3.0 to OSC 3.3 3/13 [Release 3.0 to 3.3]
Sun Storage D2 Array - Version All Versions to All Versions [Release All Releases]
Sun Storage D1000 Array - Version All Versions to All Versions [Release All Releases]
Sun Storage 3310 Array - Version Not Applicable and later
Oracle Solaris on SPARC (64-bit)
Oracle Solaris on SPARC (32-bit)
Oracle Solaris on x86-64 (64-bit)
Oracle Solaris on x86 (32-bit)

Goal

This article explains how to change a SCSI JBOD (Just A Bunch Of Disks) under Veritas Volume Manager 3.x in Solaris Cluster 3.x (D1000, D2, 3310 JBOD...)

For Solaris Volume Manager (also known as Solstice DiskSuite), refer to <Document 1004951.1> .

Solution

In the following example, c3t11d0 is a failed disk and Veritas volume manager (VxVM) has already detached it.

1. Identify the failed disk's sd instance number. Save this information for step 7.
Do this on both nodes as the sd instance number can be different between nodes.

% /usr/bin/ls -l /dev/dsk/c3t11d0s2
lrwxrwxrwx 1 root root 44 May 23 13:48
/dev/dsk/c3t11d0s2 -> ../../devices/pci@8,700000/scsi@5,1/sd@b,0:c

% /usr/bin/grep \/pci@8,700000\/scsi@5,1\/sd@b,0 /etc/path_to_inst
"/node@1/pci@8,700000/scsi@5,1/sd@b,0" 306 "sd"

2. Use scdidadm to extract information about the failed disk's DID number and diskid. Save this information for step 8.

% /usr/cluster/bin/scdidadm -L |grep c3t11d0
13 m1lab50:/dev/rdsk/c3t11d0 /dev/did/rdsk/d13
13 m1lab51:/dev/rdsk/c3t11d0 /dev/did/rdsk/d13

% /usr/cluster/bin/scdidadm -o diskid -l c3t11d0
46554a495453552030333537383031372020202000000000

or which is more readable for on-site engineers

% /usr/cluster/bin/scdidadm -o asciidiskid -l d13
IBM B3JA6038

3. On both nodes, remove the Veritas volume manager entry of the failed disk.

% /usr/sbin/vxdisk offline c3t11d0
% /usr/sbin/vxdisk rm c3t11d0
% /usr/sbin/vxdisk list | /usr/bin/grep c3t11d0
- - disk20 testdg failed was:c3t11d0s2

4. Physically disconnect the failed disk from JBOD array. Refer to the Sun System Handbook for additional instructions.

5. On both nodes, unconfigure the failed disk.

% /usr/sbin/cfgadm -c unconfigure c3::dsk/c3t11d0
% /usr/sbin/devfsadm -C

(c3t11d0 has been unconfigured as below)
% /usr/sbin/cfgadm -al |grep c3t11d0
c3::dsk/c3t11d0 unavailable connected unconfigured unknown

(device has been removed as below)
% /usr/bin/ls /dev/dsk/c3t11d0*
No match

% /usr/bin/ls /devices/pci@8,700000/scsi@5,1/sd@b,0:c
/devices/pci@8,700000/scsi@5,1/sd@b,0:c: No such file or directory

6. Physically connect a new disk to JBOD array.

7. On both nodes, configure the new disk.

% /usr/sbin/cfgadm -c configure c3::sd306
% /usr/sbin/devfsadm

(c3t11d0 has been configured as below)
% /usr/sbin/cfgadm -al |grep c3t11d0
c3::dsk/c3t11d0 disk connected configured unknown

(device has been created as below)
% /usr/bin/ls /dev/dsk/c3t11d0*
/dev/dsk/c3t11d0s0@ /dev/dsk/c3t11d0s3@ /dev/dsk/c3t11d0s6@
/dev/dsk/c3t11d0s1@ /dev/dsk/c3t11d0s4@ /dev/dsk/c3t11d0s7@
/dev/dsk/c3t11d0s2@ /dev/dsk/c3t11d0s5@

% ls /devices/pci@8,700000/scsi@5,1/sd@b,0:c
/devices/pci@8,700000/scsi@5,1/sd@b,0:c

8. On both nodes, update DID.

% /usr/cluster/bin/scdidadm -R d13

(did has been updated)
% /usr/cluster/bin/scdidadm -L | /usr/bin/grep c3t11d0
13 m1lab50:/dev/rdsk/c3t11d0 /dev/did/rdsk/d13
13 m1lab51:/dev/rdsk/c3t11d0 /dev/did/rdsk/d13

% /usr/cluster/bin/scdidadm -o diskid -l c3t11d0
46554a495453552030335038323236302020202000000000

9. On both nodes, reconstruct the volume manager.

% /usr/sbin/vxdctl enable

% /usr/sbin/vxdisk list | /usr/bin/grep c3t11d0
c3t11d0s2 sliced - - error
- - disk20 testdg failed was:c3t11d0s2

10. Finally, recover the volume manager.

% /usr/sbin/vxdiskadm
then use option 5 :
5 Replace a failed or removed disk

       (VM recovery has been started)
% /usr/sbin/vxtask list
TASKID PTID TYPE/STATE    PCT PROGRESS
173           PARENT/R   0.00% 1/0(1) VXRECOVER disk20
174    174    ATCOPY/R 01.90% 0/2097152/39928 PLXATT vol02 vol02-02

Refer to the following bug for more information: Bug ID 5102351
change, disk, cluster, suncluster, volume manager, VM, scsi, JBOD, D1000, 3310, D2, multipack
Previously Published As 78075

Attachments

This solution has no attachment