In this Document
  Symptoms   Cause   Solution   References

Applies to:

Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 11.1.0.7 - Release: 10.2 to 11.1

Information in this document applies to any platform.

Symptoms

RAC Nodes running 11.1.0.7 CRS/RDBMS release.

All the rdbms instances including ASM crashes with ORA-15064,ORA-15082

<alert_opp0221.log>

-----------------------

Wed Jan 27 17:31:01 2010
Errors in file /oracle/dumps/diag/rdbms/opp022/opp0221/trace/opp0221_asmb_27939.trc:
ORA-15064: communication failure with ASM instance
ORA-03135: connection lost contact
Process ID:
Session ID: 531 Serial number: 61
ASMB (ospid: 27939): terminating the instance due to error 15064

<alert_ASM1.log>
---------------------

Wed Jan 27 17:30:23 2010
Errors in file /oracle/asm/diag/asm/+asm/+ASM1/trace/+ASM1_ckpt_6655.trc:
ORA-15082: ASM failed to communicate with database instance
CKPT (ospid: 6655): terminating the instance due to error 15082
Wed Jan 27 17:30:24 2010
System State dumped to trace file /oracle/asm/diag/asm/+asm/+ASM1/trace/+ASM1_diag_6625.trc
Wed Jan 27 17:30:24 2010
ORA-1092 : opitsk aborting process
Wed Jan 27 17:30:24 2010

< +ASM1_ckpt_6655.trc >
-------------------------


*** ACTION NAME:() 2010-01-27 17:25:50.898

kfnmsg_SendtoUFGs: KSR failure reaping message (0xffffff83)

*** 2010-01-27 17:25:50.921
Communication failure sending message to UFGs (0xffffff83)
kfnmsg_SendtoUFGs: opcode=16 msg=397ec2bb8 local=1 client=0
2010-01-27 17:26:21.065452 : kfnmsg_SendtoUFGs: (kfnmsg) opcode=16 (KFNMS_MAPRESIZE)
2010-01-27 17:26:21.065745 : kfnmsg_SendtoUFGs: (fd) mid 0, gn [2.4294967295], fn [265.4294967295]
2010-01-27 17:26:21.065781 : kfnmsg_SendtoUFGs: (fd) extents 0, start 0, count 1456
CommunicationFailure 0xffffff83 (timeout after 300s)
..

ORA-15082 at kfnmsg_SendtoUFGs12
----- Abridged Call Stack Trace -----
<-ksedsts()+1228<-kfnmsg_SendtoUFGs()+2400<-kfnmFdOp()+348<-kffMapMesgCIC()+1256<-ksbmsg()+10096<-ksbcti()+560<-ksbabs()+980<-ksbrdp()+1168<-opirip()+1084<-opidrv()+748<-sou2o()+80<-main()+440<-_start()+380----- End of Abridged Call Stack Trace -----
error 15082 detected in background process
ORA-15082: ASM failed to communicate with database instance

Cause

This is reported in Bug 8764174 reported in 11.1.0.7
Abstract: DATABASE FAILURE DUE TO COMMUNICATION ISSUE BETWEEN DATABASE AND ASM

Bug 8764174 is closed as duplicate of Base Bug 6851110 Abstract: ASMB PROCESS HAS GROWIN IN SIZE AND IS NOT RELEASING MEMORY.

Solution

bug 6851110 is fixed in following versions-

  11.2.0.1 (Base Release)
  11.1.0.7.1 (Patch Set Update)
  10.2.0.5 (Server Patch Set)
  11.1.0.7 Patch 11 on Windows Platforms
  11.1.0.7 RAC Recommended Patch Bundle #1
  11.1.0.6 Patch 11 on Windows Platforms

Interim patch 6851110 exists for certain platform/version.

Workaround

Restart the database instance

References

BUG:6851110 - ASMB PROCESS HAS GROWIN IN SIZE AND IS NOT RELEASING MEMORY.
NOTE:735180.1 - ASMB process grows raising ora-4030 intermittently