Oracle Exadata Storage Server Software
 
Exadata Database Machine X2-2 Half Rack
 
Exadata Database Machine X2-2 Hardware
 
Exadata Database Machine X2-2 Full Rack
 
Exadata Database Machine X2-8
 
Exadata Database Machine X2-2 Qtr Rack
 
Oracle Exadata Hardware
 

PLA-Support>Eng Systems>Exadata/ODA/SSC>Oracle Exadata>DB: Exadata_EST
 

Applies to:

Symptoms

Cell  image  versions lower than 11.2.3.2.0

MS process crashed and got restarted automatically .  Cell alert log had  RS-7445 [Serv MS is absent] [It will be restarted] [] [] [] [] [] [] [] [] [] [] signalling the restart

+ No obvious errors in the ms-odl.log /cell alert log and the incident (rs*) traces on why the MS crashed ,  apart from the RS-7445 signalling the detection of its absence.

+ Callstack in incident trace shows a very generic stack:

Problem Key: RS 7445
Error: RS-7445 [Serv MS is absent] [It will be restarted] [] [] [] [] [] [] [] [] [] []
[00]: dbgePostErrorDirect [diag_dde]
[01]: ossrsutl_dump_incident []<-- Signaling
[02]: ossrsutl_monitor_srvc []
[03]: ossrsutl_monitor_srvc_prc []
[04]: sossrs_prc_start []
[05]: ossrsutl_monitor_monpr_thd []
[06]: start_thread []
[07]: clone []
[08]: 0000000000000000 []

+ Reviewing the /var/log/oracle/deploy/hs_err_pid<PID #>.log

Stack: [0x0000000040b8d000,0x0000000040c8e000),  sp=0x0000000040c8c540,  free space=1021k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x65099e]
V  [libjvm.so+0x56163b]
V  [libjvm.so+0x38612b]
V  [libjvm.so+0x3aa15b]
C  [libmsosscomm11.so+0x2fdc]                                                                                                                                             <<<<<<<<<<
C  [libmsosscomm11.so+0x142e]  Java_oracle_ossmgmt_ms_core_MSOSSComm_static_1sendrecv+0x1a2
j  oracle.ossmgmt.ms.core.MSOSSComm.static_sendrecv(I[CLjava/lang/Object;)I+0
j  oracle.ossmgmt.ms.core.MSOSSComm.getOSSMetrics(Loracle/ossmgmt/ms/core/OSSMetricList;Loracle/ossmgmt/ms/core/Position;)I+66

Cause

<Bug 11903713> - CELL-2628 DURING LOOP TEST OF CELLCLI LIST QUERIES
 

Solution

1. Ignore the error as MS will be automatically restarted upon crash, and this will not affect any functionality

2. This bug is fixed in Exadata Storage Server 11.2.3.2.1.

If on 11.2.3.2.0,  the fix included in patch 16042459 can solve the issue in many case. For further details see Exadata Critical Issues Mos note - Issue EX11

Nevertheless on 11.2.3.2.0, it may be possible to encounter this issue even if patch 16042459 is installed.  In that case, please upgrade to 11.2.3.2.1.
 

References

This solution has no attachment