某客户的数据库系统出现运行时突然宕机,查看数据库后台alert日志,无明显的ORA-00600或ORA-07445等错误;深入分析后,问题原因为存放业务数据的表空间使用满后,业务数据的表使用了LOB字段,触发了BUG:Bug 32653669 - ORA-1652 ON GEN0 CRASHED INSTANCE,导致数据库实例宕机。
分析此BUG的出发条件:1.表空间使用满 2.使用LOB字段(严格来说是securefile lob,11G后默认此种类型)。
此BUG影响范围大,在19.12.0.0.DBRU:210720版本修复。
如下为相关日志及分析:
1.数据库后台ALERT日志
2023-05-16T14:51:32.157921+08:00
ORA-1653: unable to extend table test.aaaaa by 128 in tablespace test_DATA
2023-05-16T14:51:32.161679+08:00
ORA-1691: unable to extend lobsegment test.SYS_LOB0000073703C00008$$ by 128 in tablespace test_DATA (ospid 155184)
2023-05-16T14:51:32.213277+08:00
ORA-1652: unable to extend temp segment by 128 in tablespace test_DATA
2023-05-16T14:51:32.214500+08:00
Errors in file /oracle/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_gen0_153851.trc:
ORA-01652: unable to extend temp segment by 128 in tablespace test_DATA
Errors in file /oracle/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_gen0_153851.trc (incident=240057):
ORA-1652 [] [] [] [] [] [] [] [] [] [] [] []
Incident details in: /oracle/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_240057/orcl1_gen0_153851_i240057.trc
2023-05-16T14:51:32.425261+08:00
ORA-1653: unable to extend table test.aaaaa by 128 in tablespace test_DATA
ORA-1653: unable to extend table test.aaaaa by 128 in tablespace test_DATA
ORA-1653: unable to extend table test.aaaaa by 128 in tablespace test_DATA
ORA-1653: unable to extend table test.aaaaa by 128 in tablespace test_DATA
2.当前数据库版本-19.10
[oracle@aaaaa-bak ~]$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Tue May 16 19:29:11 2023
Version 19.10.0.0.0
Copyright (c) 1982, 2020, Oracle. All rights reserved.
Connected to an idle instance.[oracle@aaaaa-bak ~]$ opatch lsinv
Oracle 临时补丁程序安装程序版本 12.2.0.1.24
版权所有 (c) 2023, Oracle Corporation。保留所有权利。
Oracle 主目录 :/oracle/app/product/19.3.0/db_1
主产品清单:/oracle/oraInventory
来自 :/oracle/app/product/19.3.0/db_1/oraInst.loc
OPatch 版本 :12.2.0.1.24
OUI 版本 :12.2.0.7.0
日志文件位置:/oracle/app/product/19.3.0/db_1/cfgtoollogs/opatch/opatch2023-05-16_19-29-16下午_1.log
Lsinventory Output file location : /oracle/app/product/19.3.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2023-05-16_19-29-16下午.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: aaaaa-bak
ARU platform id: 226
ARU platform description:: Linux x86-64
已安装的顶级产品 (1):
Oracle Database 19c 19.0.0.0.0
此 Oracle 主目录中已安装 1 个产品。
临时补丁程序 (2) :
Patch 32222571 : applied on Fri Apr 15 16:45:58 CST 2022
Unique Patch ID: 24017129
Patch description: "OCW Interim patch for 32222571"
Created on 15 Jan 2021, 01:49:46 hrs PST8PDT
Bugs fixed:
17787434, 25736599, 26305045, 26675491, 26813848, 27092095, 27148384
3.GEN进程TRACE信息
检查GEN0进程的TRACE文件,对call stacks进行分析,与ORACLE相应BUG进行对比,匹配BUG:Bug 32653669 - ORA-1652 ON GEN0 CRASHED INSTANCE