Oracle ZFS Storage Appliance: How To Correct a Boot Disk Or System Disk That Did Not Resilver After Replacement

Asset ID:	1-72-2118665.1
Update Date:	2016-09-16
Keywords:

Solution Type Problem Resolution Sure

Solution 2118665.1 : Oracle ZFS Storage Appliance: How To Correct a Boot Disk Or System Disk That Did Not Resilver After Replacement

Applies to:

Oracle ZFS Storage ZS3-4 - Version All Versions to All Versions [Release All Releases]
Oracle ZFS Storage ZS4-4 - Version All Versions to All Versions [Release All Releases]
Sun Storage 7210 Unified Storage System - Version All Versions to All Versions [Release All Releases]
Sun Storage 7410 Unified Storage System - Version All Versions to All Versions [Release All Releases]
Sun ZFS Storage 7420 - Version All Versions to All Versions [Release All Releases]
7000 Appliance OS (Fishworks)

Symptoms

Here we have replaced a system boot disk and it has not worked:

    NAME       STATE    READ WRITE CKSUM
    system     DEGRADED    0 0 0
    mirror-0   DEGRADED    0 0 0
      c2t1d0s0 ONLINE      0 0 0
      c2t0d0s0 UNAVAIL     0 0 0 cannot open or corrupted data

The drive is seen in format though:

1. c2t0d0 <ATA -HITACHI HTE5450-C70E cyl 60799 alt 2 hd 255 sec 63> drive ok
/pci@1,0/pci10de,cb84@5,1/disk@0,0
2. c2t1d0 <ATA -HITACHI HTE5450-C70E cyl 60798 alt 2 hd 255 sec 63>
/pci@1,0/pci10de,cb84@5,1/disk@1,0

So we have the new disk inserted and nothing happened.

You may have rebooted the head to get Solaris to see the device, as the appliance needs the device to be seen there

In any event, the reboot still did not kick of a resilver nor did killing svc.startd

# pgrep svc.startd
244
# kill 244

Also you have cleared FMA down as well.

Cause

Various reasons - normally seen on 7x10 series due to 'nvsata' bug.

Solution

If you're happy with the disk number from (MDB) ::spa -v or just CxTxDx then:

# zpool detach system c2t0d0s0

# zpool detach system 17735178614861284062 (here we used the UUID )

Now you see this :

    NAME       STATE READ WRITE CKSUM
    system     ONLINE   0 0 0
      c2t1d0s0 ONLINE   0 0 0

OK, put the new drive back into the pool - physically its been there since the reboot of course.

Best to used the aklabel command:

# aklabel -g /dev/rdsk/c2t0d0s0

# aklabel is not to be used on data drives, only on drives from the system pool.

It creates an SMI label and data drives have an EFI label.

I checked format and saw the correct disk there and used:

# zpool attach -f <pool> live_disk new_disk

# zpool attach -f system c2t1d0s0 c2t0d0s0

-f to force - you'll need it.

Now check that it resilvers OK:

    NAME S       TATE   READ WRITE CKSUM
    system       ONLINE    0 0 0
      mirror-0   ONLINE    0 0 0
        c2t1d0s0 ONLINE    0 0 0
      c2t0d0s0 ONLINE    0 0 0 (resilvering)

Attachments

This solution has no attachment