ORA-00600: internal error code, arguments: [kcblasm_1], [103], [], [], [], [], [], []

 

• 环境描述

 

 

AIX5.3  HACMP  Oracle 10.2.0.5

 

 

• 故障描述

 

 

Sun Jun 01 06:46:43 BEIST 2014

 

 

Errors in file /u01/app/admin/findbb/udump/findbb_ora_696582.trc:

 

 

ORA-00600: internal error code, arguments: [kcblasm_1], [103], [], [], [], [], [], []

 

 

• findbb_ora_696582.trc内容

 

 

Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /u01/app/product/10.2.0/db_1
System name:	AIX
Node name:	findbb2
Release:	3
Version:	5
Machine:	00CBE0F64C00
Instance name: findbb
Redo thread mounted by this instance: 1
Oracle process number: 78
Unix process pid: 614538, image: oracle@findbb2

*** 2014-06-01 06:03:58.691
*** SERVICE NAME:(SYS$USERS) 2014-06-01 06:03:58.670
*** SESSION ID:(403.23176) 2014-06-01 06:03:58.670
WARNING:Could not lower the asynch I/O limit to 4128 for SQL direct I/O. It is set to -1 
*** 2014-06-01 08:19:03.172
ksedmp: internal or fatal error
ORA-00600: internal error code, arguments: [kcblasm_1], [103], [], [], [], [], [], []
Current SQL statement for this session:
MERGE INTO DS_RB_ACCT_TIME_TMP1 DES USING (SELECT INTERNAL_KEY,

 

• 临时故障处理

是由于hash join的bug增加如下即可

增加至父存储过程的begin后面，也可以增加至该语句的存储过程。
execute immediate 'alter session set "_hash_join_enabled"= false';

建议在存储过程末尾增加关闭操作
execute immediate 'alter session set "_hash_join_enabled"= true';
也建议增加如下
execute immediate 'alter session set "_gby_hash_aggregation_enabled"=false';

• 补丁升级处理

参考note 1133845.1

$ opatch lsinv
Invoking OPatch 10.2.0.4.9

Oracle Interim Patch Installer version 10.2.0.4.9
Copyright (c) 2009, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/product/10.2.0/db_1
Central Inventory : /home/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 10.2.0.4.9
OUI version       : 10.2.0.5.0
OUI location      : /u01/app/product/10.2.0/db_1/oui
Log file location : /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-07-25_19-44-07PM.log

Patch history file: /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt

Lsinventory Output file location : /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2014-07-25_19-44-07PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (3): 

Oracle Database 10g                                                  10.2.0.1.0
Oracle Database 10g Release 2 Patch Set 3                            10.2.0.4.0
Oracle Database 10g Release 2 Patch Set 4                            10.2.0.5.0
There are 3 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


--------------------------------------------------------------------------------

OPatch succeeded.
$ cd yuanding
$ ls
7612454                                         p7612454_10205_AIX64-5L.zip
dba_snapshot_database_10g.sql                   p8202632_10205_AIX64-5L_2of2.zip
dba_snapshot_database_10g_FINDBB_20131011.html
$ cd 7612454
$ ls
README.txt  etc         files
$ opatch apply
Invoking OPatch 10.2.0.4.9

Oracle Interim Patch Installer version 10.2.0.4.9
Copyright (c) 2009, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/product/10.2.0/db_1
Central Inventory : /home/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 10.2.0.4.9
OUI version       : 10.2.0.5.0
OUI location      : /u01/app/product/10.2.0/db_1/oui
Log file location : /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-07-25_19-44-55PM.log

Patch history file: /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt

ApplySession applying interim patch '7612454' to OH '/u01/app/product/10.2.0/db_1'

Running prerequisite checks...

OPatch detected non-cluster Oracle Home from the inventory and will patch the local system only.


Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/product/10.2.0/db_1')


Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files and inventory (not for auto-rollback) for the Oracle Home
Backing up files affected by the patch '7612454' for restore. This might take a while...
Backing up files affected by the patch '7612454' for rollback. This might take a while...

Patching component oracle.rdbms, 10.2.0.5.0...
Updating archive file "/u01/app/product/10.2.0/db_1/lib/libserver10.a"  with "lib/libserver10.a/kcbl.o"
Running make for target ioracle
ApplySession adding interim patch '7612454' to inventory

Verifying the update...
Inventory check OK: Patch ID 7612454 is registered in Oracle Home inventory with proper meta-data.
Files check OK: Files from Patch ID 7612454 are present in Oracle Home.

The local system has been patched and can be restarted.


OPatch succeeded.
$ opatch lsinv
Invoking OPatch 10.2.0.4.9

Oracle Interim Patch Installer version 10.2.0.4.9
Copyright (c) 2009, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/product/10.2.0/db_1
Central Inventory : /home/oracle/oraInventory
   from           : /etc/oraInst.loc
OPatch version    : 10.2.0.4.9
OUI version       : 10.2.0.5.0
OUI location      : /u01/app/product/10.2.0/db_1/oui
Log file location : /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-07-25_19-45-58PM.log

Patch history file: /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txt

Lsinventory Output file location : /u01/app/product/10.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2014-07-25_19-45-58PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (3): 

Oracle Database 10g                                                  10.2.0.1.0
Oracle Database 10g Release 2 Patch Set 3                            10.2.0.4.0
Oracle Database 10g Release 2 Patch Set 4                            10.2.0.5.0
There are 3 products installed in this Oracle Home.


Interim patches (1) :

Patch  7612454      : applied on Fri Jul 25 19:45:31 GMT+08:00 2014
Unique Patch ID:  12780900
   Created on 22 Jul 2010, 03:58:28 hrs PST8PDT
   Bugs fixed:
     7612454

--------------------------------------------------------------------------------

OPatch succeeded.

 

• 备注

In this Document

Symptoms

Changes

Cause

Solution

References

---

 

APPLIES TO:

Oracle Database - Enterprise Edition - Version 10.2.0.5.0 to 10.2.0.5.0 [Release 10.2]
Information in this document applies to any platform.
***Checked for relevance on 18-Jul-2013***

SYMPTOMS

Hitting ORA-600 [kcblasm_1] in 10.2.0.5:

In the ALERT:

ORA-00600: internal error code, arguments: [kcblasm_1], [103], [], [], [], [], [], []

The call stack in the generated tracefiles looks similar to:

kcblasm1 kcblasm kxhfFndFreeSlot kxhfNewBuffer qerhjGetNewBuffer ksxb1bqb kxhrPack

 

CHANGES

This problem is hit only in 10.2.0.5, up to PSU 10.2.0.5.4 in which problem is fixed.
Problem is affecting 10.2.0.5 to 10.2.0.5.3.

CAUSE

This is is a known and common problem hit in 10.2.0.5, investigated and corrected in unpublished Bug 7612454.

The problem was introduced in 10.2.0.5, by the fix of Bug:7523755.
This has been corrected in PSU 10.2.0.5.4 and 11.2.

 

SOLUTION

 To quickest way to the solution of this problem is to apply PSU 10.2.0.5.4, Patch 12419392 or later.

All alternative solutions for this problem are listed below:

• Upgrade the database to 11.2.
  
  OR
   
• Apply 10.2.0.5.4 Patch Set Update (Patch 12419392) or later PSUs where bug is fixed. 
  The available PSUs are mentioned in "10.2.0.5 Patch Set Updates - List of Fixes in each PSU" (Document 1337394.1)
  
  OR
   
• Apply interim Patch 7612454 on top of 10.2.0.5 (10.2.0.5.0-10.2.0.5.3):
   
  • For UNIX / Linux platforms apply Patch 7612454 available for download on MOS.
  • For Windows platforms apply Patch 3 or higher. 
    Please check Document 342443.1 for latest patches available for Windows on top of 10.2.0.5.

Note: 
It is possible to hit ORA-600 [kcblasm_1] in 10.2.0.4 or 11.1.0.7 if an early version of the one-off Patch 7523755 was applied. 
The problem patch Patch 7523755 was quickly removed and a correct version of the patch is since long available for download at MOS.
For details of the problem of hitting ORA-600 [kcblasm_1] in 10.2.0.4 or 11.1.0.7, please take part of Document 849381.1.