Oracle Database Appliance X4-2
 
Oracle Database Appliance Software
 
Oracle Database Appliance
 
Oracle Database Appliance X3-2
 

PLA-Support>Sun Systems>x86>Engineered Systems HW>SN-x64: ORA-DATA-APP
 

In this Document

Symptoms

Changes

Cause

Solution

References

Created from <SR 3-5334656327>

Applies to:

Symptoms

    FOR SUPPORT ONLY - there are many flavors of this problem across several ODA versions:    Please review INTERNAL information at the bottom of those note for further discussion and information 

As a result of randomly missing disks, ASM disks or groups do not come up:

• ODA and ASM is failing to identify disks during startup
• The problem symptom includes different disks randomly showing a failed,  predictive failure or missing
• The problem can lead to an entire node failing to startup

Example:

Changes

This problem can occur after:

• One ASM disk is lost
• Replacing a disk
• Reboot (of one server)

Cause

 
#2 There are other similar scenarios for finding disks offlined that are not the result of a specific bug

   CR 7132662 - P1 erie/firmware Cluster outage resulted from a single HDD failure - X4370M2 with Erie

Solution

Resolution for #1:

See Urgent Mandatory OAK Patch 2.1.0.3.1 <Document: 1438089.1>

Workaround
------------
1) Power-cycle both (2) of the servers

Resolution for #2

Bug 17387042 : LNX64-112-CMT: OAKCLI SHOW DISK SHOULD GIVE A WARNING/ERROR IF A DISK IS MISSING (2.8)
- Add cross reference Bug 18418872 : DIAGNOSABILITY IMPROVEMENTS
Non-ODA
ASM Is Not Discovery/Detecting New Candidate Disks On Only One RAC Node Of Several Nodes In The Cluster (Too many open files). (Doc ID 1548607.1) -- Version 10.2.0.1 to 11.2.0.4 Any Platform
 
ODA (Oracle Database Appliance) Different Disks Randomly Disappear After a Reboot (Doc ID 1420126.1)
ALERT - Urgent Mandatory OAK Patch 2.1.0.3.1 for ODA - (Oracle Database Appliance) (Doc ID 1438089.1)
ODA (Oracle Database Appliance) troubleshooting and solutions for ORA-600 [kfdjoin3] causing ASM startup failure after patching to 2.5 or 2.6 (Doc ID 1557502.1)
Replaced ODA drive lists as “UNKNOWN PARTIAL PathsNotLoaded” (Doc ID 1536486.1)
ODA : Disk space usage on boot mount point shows warnings such as Disk Space Usage on mount point: "/boot is (XX%)" (Doc ID 1537133.1)
Bug 17387042 : LNX64-112-CMT: OAKCLI SHOW DISK SHOULD GIVE A WARNING/ERROR IF A DISK IS MISSING
Bug 16231699 - Adding a new disk on ODA appliance hits ORA-600 [kfgpset3] (Doc ID 16231699.8)
Bug 18740794 : ODA SHOW DISK STATUS IS GOOD HOWEVER DISK IS OFFLINED (2.7)
In the same category, disk failed, oakcli show good online.
Bug 18477093 - OAKCLI SHOW DISK SAYS GOOD ONLINE, IN FACT SMART FAILURE

Bug 18418872 : LNX64-112-CMT: DIAGNOSABILITY IMPROVEMENTS
 Dup of bug 17387042: OAKCLI SHOW DISK SHOULD GIVE A WARNING/ERROR IF A DISK  IS MISSING

ODA HW: After SSD replacement, disk shows STATE as UNINITIALIZED and STATE_DETAILS as NewDiskInserted (Doc ID 1461219.1)
bug 16803770 - unable to get rid of uninitialized newdiskinserted

ODA (Oracle Database Appliance): ORA-00600: [kfdApplianceDiskNum0] adding a disk (Doc ID 1457115.1)
The configuration file: /opt/oracle/extapi/asmappl.config  is missing on the right disk entries
Edit     /opt/oracle/extapi/asmappl.config

Add the missing entries  (EXAMPLE for HDD  S13)
    disk  /dev/mapper/HDD_E0_S13_1137808215p1       0  13  1
    disk  /dev/mapper/HDD_E0_S13_1137808215p2       0  13  2

Then   alter diskgroup /*+ _OAK_AsmCookie */ [DATA | RECO | REDO] add disk.....

 You can check as ASM is running the "rebalance":

SQL> select GROUP_NUMBER, OPERATION, STATE, ACTUAL, SOFAR, EST_MINUTES from v$asm_operation;


2.1.x
Bug 13728921 : PHYSICAL DISKS DISAPPEAR AFTER REBOOTING NODE 2
96 - Closed, Duplicate Bug -- Bug 13618428
Bug 13370690 : LNX64-112-CMT:DISK MISSED AT OS AFTER REBOOT (2.1.0.0.0)

#1 Please 'oakcli update -patch 2.1.0.0.0' to update firmware version to eliminate the known issues.
@ For #2, disk 10 missing and reappearing after re-seated.
@ This is Seagate FW problem.  
@ CR 7132662 - P1 erie/firmware Cluster outage resulted from a single HDD
@ failure - X4370M2 with Erie
@ .
@ Before Seagate can provide us a fix,
@ we will provide a workaround to reduce disk queue_depth in 2.1.0.3.1.
@ Workaround by oracle is uploaded to 2.1.0.3.1 place holder Bug 13817532.
@   -- Bug 13812384 - QUEUE DEPTH TO BE SET FOR SHARED DISKS


Bug 13618428 : AFTER LOSING ONE ASM DISK, MULTIPLE DISKS BECAME UNRESPONSIVE

All disks are actually not bad.
@
@ If you are close to the lab, then you can reseat the disk.   
@ If you lost slot 3, issue 'oakcli locate pd_03 on', led lights should be
on.
@ Then remove the disks, wait 1-2 minutes, insert the disks.  No downtime.   
@ Or if you're not close to the lab, then using ilom to power down
@ the machine, wait 1-2 minutes, power up the machine.  
@ All the missing disks will re-appear.
@ .
@ This is Seagate FW problem.  Before they can provide us a fix,
@ we will provide a workaround to reduce disk queue_depth in 2.1.0.3.1.
@ Workaround will be uploaded to 2.1.0.3.1 place holder Bug 13817532

References

This solution has no attachment