Exadata X3-8 Hardware
 
Exadata Database Machine X2-2 Hardware
 

PLA-Support>Eng Systems>Exadata/ODA/SSC>Oracle Exadata>DB: Exadata_EST
 

Incorrect Celldisk Disk Name found for three flashdisks in storage cell after flashcards were previously installed on another storage cell and then moved to another storage cell

In this Document

Symptoms

Cause

Solution

Created from <SR 3-6962714031>

Applies to:

Symptoms

Found flashcache degraded as it was short 3 celldisks.

When reviewing the flashdisks on cell 'sl08cel02' found flashdisks 4,5 & 6 mis-labelled as 'sl07cel07' instead of 'sl07cel08'.

Cause

The flashcards on pci slot 2 have wrong name. This only happens if the flashcards were used on a different storage cell.
The name is similar cel07 where the correct name is cel02.
 
It looks like they were used on a different storage cell from the same customer.

The devices affected are:

/dev/sdz, /dev/sdaa, /dev/sdab, /dev/sdac.
 

Solution

for i in /dev/sdz /dev/sdaa /dev/sdab /dev/sdac
do
  dd if=$i bs=8k count=32|strings|grep cel07
done

It will return an output similar to

32+0 records in
32+0 records out
262144 bytes (262 kB) copied, 0.00436014 seconds, 60.1 MB/s
FD_04_cel07
32+0 records in
32+0 records out
262144 bytes (262 kB) copied, 0.00443345 seconds, 59.1 MB/s
FD_05_cel07
32+0 records in
32+0 records out
262144 bytes (262 kB) copied, 0.00444593 seconds, 59.0 MB/s
FD_06_cel07
32+0 records in
32+0 records out
262144 bytes (262 kB) copied, 0.00342453 seconds, 76.5 MB/s
FD_07_cel07

The celldisk are created based on the metadata found on the flashdisk, that's why the celldisk name references cell cel07.

SOLUTION

1. Drop the celldisks affected

cellcli -e drop celldisk FD_04_cel07
cellcli -e drop celldisk FD_05_cel07
cellcli -e drop celldisk FD_06_cel07
cellcli -e drop celldisk FD_07_cel07

2. Use following command to clear the metadata


for i in /dev/sdz /dev/sdaa /dev/sdab /dev/sdac
do
  dd if=/dev/zero of=$i bs=1M count=20
done


3. Create the celldisks

cellcli -e create celldisk all flashdisk

This will skip the celldisks on the flashdisks already created.


4. Validate the celldisks are created with the correct name

cellcli> list celldisk where devicename='/dev/sdz' detail
cellcli> list celldisk where devicename='/dev/sdaa' detail
cellcli> list celldisk where devicename='/dev/sdab' detail
cellcli> list celldisk where devicename='/dev/sdac' detail

If cell is running FlashCacheMode in WriteTrhough, then:

6. Drop and recreate the flashcache

cellcli -e drop flashcache
cellcli -e create flashcache all

If cell is running FlashCacheMode in WriteBack, then

Validate the current configuration of flascache

     name:                   cel02_FLASHCACHE
     cellDisk:               FD_08_cel02,FD_01_cel02,FD_00_cel02,FD_12_cel02,FD_03_cel02,FD_07_cel02,FD_11_cel02,FD_10_cel02,FD_15_cel02,FD_09_cel02,FD_13_cel02,FD_14_cel02,FD_02_cel02
     creationTime:           2012-07-17T17:01:22-07:00
     degradedCelldisks:     
     effectiveCacheSize:     296.359375G
     id:                     36a266ea-28b2-4906-bbb9-afe13c97e1ca
     size:                   296.359375G
     status:                 normal

In this example, the flashcache was configured,  in order to add the remaining flash disks, following command will reconfigure the flashcache to the full capacity of 16 flashdisks:

Validate full capacity running command

Validate all the griddisks are cached, running command

 If there are issues and entire flashcache needs to be dropped, then execute following commands

• Validate the griddisks are normal

• Flush the dirty cache to disk

• Validate the status of flush

 Attribute flushstatus should return "completed" for all celldisks, to be ready to drop flashcache

• Drop the flashcache

• Create flashcache

• Validate all the griddisks are cached, running command

This solution has no attachment