Sun Microsystems, Inc.  Oracle System Handbook - ISO 7.0 May 2018 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-72-1646106.1
Update Date:2017-10-04
Keywords:
Solution Type  Problem Resolution Sure

Solution  1646106.1 :   Node Reboot with clssscExit: CSSD signal 11 in thread GMClientListener  

Related Items 
    

  • Oracle Database - Enterprise Edition
    •  
  • Exadata Database Machine X2-2 Hardware
    •  
Related Categories 
    

  • PLA-Support>Database>DB Systems>Scalability>DB: RAC
    •  






In this Document


Symptoms


Cause


Solution


References

Created from <SR 3-8669207211>


Applies to:  

Exadata Database Machine X2-2 Hardware - Version All Versions and later
Oracle Database - Enterprise Edition - Version 11.2.0.3 to 11.2.0.4 [Release 11.2]
Information in this document applies to any platform.



Symptoms


DB Node reboot with below error message in $GRID_HOME/log/<host>/cssd/ocssd.log:


2014-03-08 01:38:46.039: [    CSSD][1102399808]clssgmFenceCompletion: (0x7ff19852d320) process death fence completed for process 17436, object type 2
2014-03-08 01:38:46.039: [    CSSD][1102399808]clssgmTermShare: (0x7ff194651f50) global grock DBRACDB member 2 type 1
2014-03-08 01:38:46.039: [    CSSD][1102399808]clssgmUnreferenceMember: global grock DBRACDB member 2 refcount is 57
2014-03-08 01:38:46.041: [    CSSD][1075673408]clssgmFenceClient: fencing client (0x7ff195051ec0), member 0 in group DG_LOCAL_DATA, same group share, death fence 0, SAGE fence 1
2014-03-08 01:38:46.045: [    CSSD][1075673408]###################################
2014-03-08 01:38:46.045: [    CSSD][1075673408]clssscExit: CSSD signal 11 in thread GMClientListener
2014-03-08 01:38:46.045: [    CSSD][1075673408]###################################
2014-03-08 01:38:46.045: [    CSSD][1075673408](:CSSSC00012:)clssscExit: A fatal error occurred and the CSS daemon is terminating abnormally
2014-03-08 01:38:46.059: [    CSSD][1075673408]

----- Call Stack Trace -----
2014-03-08 01:38:46.059: [    CSSD][1075673408]calling              call     entry                argument values in hex      
2014-03-08 01:38:46.059: [    CSSD][1075673408]location             type     point                (? means dubious value)     
2014-03-08 01:38:46.059: [    CSSD][1075673408]-------------------- -------- -------------------- ----------------------------
2014-03-08 01:38:46.144: [    CSSD][1075673408]clssscExit()+740     call     kgdsdst()            000000000 ? 000000000 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]                                                   0401B8468 ? 000000001 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]                                                   000000001 ? 000000003 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]s0clsssc_sighandler  call     clssscExit()         00089CA80 ? 000000002 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]()+611                                             0401B8468 ? 000000001 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]                                                   000000001 ? 000000003 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]__sighandler()       call     s0clsssc_sighandler  00000000B ? 000000002 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]                              ()                   0401B8468 ? 000000001 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]                                                   000000001 ? 000000003 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]pthread_mutex_unloc  signal   __sighandler()       000000000 ? 000F1EBD0 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]k()                                                000F1EBD0 ? 000000000 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.144: [    CSSD][1075673408]sltsmnr()+8          call     pthread_mutex_unloc  000000000 ? 000F1EBD0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                              k()                  000F1EBD0 ? 000000000 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clssgmFenceClient()  call     sltsmnr()            000000000 ? 000F1EBD0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]+862                                               000F1EBD0 ? 000000000 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clssgmDeadProc()+51  call     clssgmFenceClient()  00089CA80 ? 7FF195051EC0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]9                                                  7FF194728560 ? 000000000 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clssgmclienteventhn  call     clssgmDeadProc()     00089CA80 ? 7FF194728560 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]dlr()+4730                                         7FF194728560 ? 000000000 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clssscSelect()+1623  call     clssgmclienteventhn  000000007 ? 000720AF0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                              dlr()                0401BDFE0 ? 000000000 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clssgmProcClientReq  call     clssscSelect()       00089CA80 ? 000720AF0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]s()+540                                            0000000D9 ? 000436D46 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clssgmclientlsnr()+  call     clssgmProcClientReq  00089CA80 ? 000720AF0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]1590                          s()                  0000000D9 ? 000436D46 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clssscthrdmain()+25  call     clssgmclientlsnr()   00089CA80 ? 000720AF0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]3                                                  0000000D9 ? 000436D46 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]start_thread()+221   call     clssscthrdmain()     00089CA80 ? 000720AF0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000720AF0 ? 000436D46 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]clone()+109          call     start_thread()       0401D7940 ? 000720AF0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000720AF0 ? 000436D46 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]0000000000000000     call     clone()              0401D7940 ? 000720AF0 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000720AF0 ? 000436D46 ?
2014-03-08 01:38:46.145: [    CSSD][1075673408]                                                   000613A30 ? 026FB4958 ?








Cause


This issue has been addressed in 


  <Bug 17947858> - CLSSSCEXIT: CSSD SIGNAL 11 IN THREAD GMCLIENTLISTENER


which was identified as a duplicate of unpublished


  Bug 16765834 - LNX64-11204-CSS: CSSD CRASHED WHEN MULTIPLE DB SHUTDOWN IN PARALLEL


To match this bug the call  stack in the ocssd.log should contain the following functions: clssgmFenceClient <- clssgmDeadProc.
 



Solution


Bug 16765834 has been fixed with the 11.2.0.4 patchset, interim patches for 16765834 have been provided for (on top of) various 11.2.0.3 PSUs. Please check 'Patches and Updates' on My Oracle Support for more details.  


 



Attachments

This solution has no attachment


















       

Copyright © 2018 Oracle, Inc.  All rights reserved.

 Feedback