SuperCluster: Intermittent ORA-29283: invalid file operation ORA-06512: at "SYS.UTL_FILE" when access external file or KUP-04040 or KUP-04062 from external tables

Asset ID:	1-72-1929525.1
Update Date:	2015-03-23
Keywords:

Solution Type Problem Resolution Sure

Solution 1929525.1 : SuperCluster: Intermittent ORA-29283: invalid file operation ORA-06512: at "SYS.UTL_FILE" when access external file or KUP-04040 or KUP-04062 from external tables

Applies to:

SPARC SuperCluster T4-4 - Version All Versions to All Versions [Release All Releases]
Oracle SuperCluster M6-32 Hardware - Version All Versions to All Versions [Release All Releases]
Oracle SuperCluster T5-8 Hardware - Version All Versions to All Versions [Release All Releases]
Oracle Database - Enterprise Edition - Version 11.2.0.3 to 12.1.0.1 [Release 11.2 to 12.1]
Oracle Solaris on SPARC (64-bit)

Symptoms

External file operations from the database failing due to a libc file operation such as open, stat or access being interupted by a sigalarm.

Database errors will look as such

utl_file CASE

Intermitent ORA-29283: invalid file operation ORA-06512: at "SYS.UTL_FILE" when access external file

External Tables case

Intermittant stacks such as

ERROR at line 1:
ORA-12801: error signaled in parallel query server P011, instance
<server:sid> (1)
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04040: file <filename> in EXT_DB_DIR_1 not found

ERROR at line 1:
ORA-29913: error in executing ODCIEXTTABLEOPEN callout
ORA-29400: data cartridge error
KUP-04062: no data source specified

A truss of the failing processes ( this example will take a sid name in and truss specific libc calls for every parallel query slave)

#!/usr/bin/ksh
# $1 is database name
while true
do
PIDLIST=`ps -ef | grep ora_p | grep $1 | grep -v ping | grep -v psp | grep -v pz | grep -v pmon | nawk '{print $2}'`
for i in $PIDLIST
do
    if [ ! -e truss_${i}.out ] ; then
      print "trussing" ${i}
      truss -d -t access,stat,openat,open -u 'libc:access,stat,open,openat' -p $i >> truss_${i}.out 2>&1 &
    fi
done
sleep 10
done

Changes

Cause

A database external call to a libc file operation such as open, stat or access being interupted by a sigalarm. The default behavior of Solaris is to return ERESTART as opposed to SA_RESTART in these cases. There have been two enhancements to correct this behavior they are annotated in internal bugs 18448344 and 19248820.

Solution

Apply OCT 2014 QFSDP where Jumbo idr1163.6 is included or request obtain Jumbo IDR 940.10 for SRU 11.4. Systems. The fixes will not be backported any further than that.

References

<NOTE:1632521.1> - SuperCluster- Solaris 11 Support Repository Updates (SRU) and Jumbo Interim Diagnostic Relief (JIDR) Support Matrix.
<BUG:19248820> - ACCESS & STAT DON'T COMPLY WITH POSIX DEFINITION WRT. SA_RESTART
<BUG:18448344> - OPEN(2) IS NOT COMPLYING WITH THE POSIX DEFINITION WRT. SA_RESTART IN SIGACTION

Attachments

This solution has no attachment