Getting ORA-15032 ORA-1541 And/Or ORA-15260 While Adding The Disks Into DBFS DiskGroup When Expanding The Exadata i.e. Adding New Cell(Storage) Nodes

Asset ID:	1-72-2165176.1
Update Date:	2017-03-31
Keywords:

Solution Type Problem Resolution Sure

Solution 2165176.1 : Getting ORA-15032 ORA-1541 And/Or ORA-15260 While Adding The Disks Into DBFS DiskGroup When Expanding The Exadata i.e. Adding New Cell(Storage) Nodes

Applies to:

Oracle Exadata Storage Server Software - Version 12.1.1.1.0 to 12.1.2.2.3 [Release 12.1]
Exadata X4-2 Hardware - Version All Versions to All Versions [Release All Releases]
Information in this document applies to any platform.

Symptoms

If you are getting below errors while adding the disks into the DBFS DiskGroup on Exadata

This situation may occur while expanding the Exadata i.e. adding CELL(Storage) nodes.

In this example ; we are adding 4 NEW CELL (Storage) Nodes into the Existing Rack i.e. Quarter Rack ( 3 Existing CELL NODES )

ERROR

=============

ORA-15032: not all alterations performed
ORA-15415: could not retrieve appliance properties for disk group DBFS_DG

Getting below errors when changing the diskgroup attribute of DBFS

ERROR

=============

SQL> alter diskgroup dbfs_dg set attribute 'content.type' = 'system'
Mon Jul 25 00:14:32 2016
ORA-15032: not all alterations performed
ORA-15260: permission denied on ASM disk group

Cause

Number of disks for DBFS diskgroup is NOT same across ALL the CELL(storage) nodes i.e. OLD cell(storage) nodes and NEWLY added cell(storage) nodes.

Older storage(cell) has 10 physical disks allotted for GRID DISKS for DBFS whereas newer CELL has 12 physical disks so there is a mismatch.

DBFS ASM Diskgroup does NOT get/use first 2 disks.

You can see query result/output below

NAME COUNT(*)
------------------------------ ----------
DATAC1 84 <<<<<<<<<<<<<<<<<<<<<<<<<<<<< 7 CELL NODES ( 12*7)

DBFS_DG 30 <<<<<<<<<<<<<<<<<<<<<<<<<<<<< 3 Cell Nodes , each with 12 physical disks so there should be 36 disks but it has only 30 disks because first 2 disks are being used by Operating System AND not available for DBFS

( DATA and RECO )

RECOC1 84 <<<<<<<<<<<<<<<<<<<<<<<<<<<<< 7 CELL NODES ( 12*7)

Solution

1) Please login to ALL NEW CELL NODES ( In Our Example we are adding 4 new CELL nodes addition to existing 3 CELL nodes )

MyCellAdm04 <<<<<<<< NEW CELL NODE
MyCellAdm05 <<<<<<<< NEW CELL NODE
MyCellAdm06 <<<<<<<< NEW CELL NODE
MyCellAdm07 <<<<<<<< NEW CELL NODE

Drop the grid disks in first two slots

Explanation
=============

Each CELL NODE has 12 Physical Disks and out of these 12 Physical Disks , first 2 disks ( slot 0 and slot 1 ) of each CELL(storage) node ; are being used by Exadata CELL Node Operating System ( Linux & Exadata Software ) and not available for DBFS diskgroup and this is the reason , DATA , RECO has 36 disks on previous CELL nodes however DBFS has using only 30 ( 10 Physical Disks * 3 CELL nodes)

==============

Drop following griddisks ONLY on NEW CELL nodes

DBFS_DG_CD_00_MyCellAdm04
DBFS_DG_CD_01_MyCellAdm04

DBFS_DG_CD_00_MyCellAdm05
DBFS_DG_CD_01_MyCellAdm05

DBFS_DG_CD_00_MyCellAdm06
DBFS_DG_CD_01_MyCellAdm06

DBFS_DG_CD_00_MyCellAdm07
DBFS_DG_CD_01_MyCellAdm07

Example on

==================

MyCellAdm04 <<<<<<<< NEW CELL NODE
MyCellAdm05 <<<<<<<< NEW CELL NODE
MyCellAdm06 <<<<<<<< NEW CELL NODE
MyCellAdm07 <<<<<<<< NEW CELL NODE

ON CELL MyCellAdm04 ; root user on cellcli prompt

ALTER GRIDDISK DBFS_DG_CD_00_MyCellAdm04 INACTIVE;
DROP GRIDDISK DBFS_DG_CD_00_MyCellAdm04 ;
alter griddisk DBFS_DG_CD_01_MyCellAdm04 inactive;
drop griddisk DBFS_DG_CD_01_MyCellAdm04 ;

Please do the same action on other CELL nodes also ( in this example MyCellAdm05 , MyCellAdm06 ,MyCellAdm07 )

2)
------------
Now add the disks into DBFS ASM Diskgroup .

alter diskgroup DBFS_DG add disk 'o/*/DBFS_DG*MyCellAdm04', 'o/*/DBFS_DG*MyCellAdm05', 'o/*/DBFS_DG*MyCellAdm06', 'o/*/DBFS_DG*MyCellAdm07' rebalance power 4 NOWAIT

Attachments

This solution has no attachment