新搭建的oracle数据库出现宕机异常，报错ORA-27157、ORA-27300、ORA-27301、ORA-27302、ORA-27303、ORA-00600...

最新推荐文章于 2025-05-28 23:39:14 发布

18767873417

最新推荐文章于 2025-05-28 23:39:14 发布

阅读量159

点赞数 1

文章标签：数据库 oracle

问题描述：新搭建的oracle数据库出现宕机异常，报错ORA-27157、ORA-27300、ORA-27301、ORA-27302、ORA-27303、ORA-00600，如下所示：
数据库：oracle 12.2.0.1 单实例
系统：rhel 7.2
1、告警日志
如下为数据库宕机时后台日志的告警日志内容.
2024-07-26T02:00:15.986490+08:00
TABLE SYS.WRI$_OPTSTAT_HISTHEAD_HISTORY: ADDED INTERVAL PARTITION SYS_P226 (45498) VALUES LESS THAN (TO_DATE(' 2024-07-27 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN'))
TABLE SYS.WRI$_OPTSTAT_HISTGRM_HISTORY: ADDED INTERVAL PARTITION SYS_P229 (45498) VALUES LESS THAN (TO_DATE(' 2024-07-27 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN'))
2024-07-26T02:05:12.061455+08:00
Resize operation completed for file# 3, fname /oracle/app/oracle/oradata/tmis/sysaux01.dbf, old size 819200K, new size 829440K
2024-07-26T02:15:08.370939+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_tt01_31413.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
2024-07-26T02:15:08.581879+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_pmon_31254.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
2024-07-26T02:15:08.651104+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_m000_31395.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
2024-07-26T02:15:08.787484+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_tt00_31409.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_pmon_31254.trc  (incident=40977):
ORA-27157 [] [] [] [] [] [] [] [] [] [] [] []
Incident details in: /oracle/app/oracle/diag/rdbms/tmis/tmis/incident/incdir_40977/tmis_pmon_31254_i40977.trc
2024-07-26T02:15:08.836687+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_lg00_31344.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1

......

2024-07-26T02:15:09.177907+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_w00c_2224.trc  (incident=41761):
ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], []
ORA-27300: OS system dependent operation:semop failed with status: 22
ORA-27301: OS failure message: Invalid argument
ORA-27302: failure occurred at: sskgpwwait2b
ORA-27303: additional information: ctx(0x7f5fe207c000); wid(0xae4b8980); flags(200)
semid(0x458004); sem_flg(0x0); sem_num(104); oldval(-1); errno(22)
ORA-00448: normal completion of background process
Incident details in: /oracle/app/oracle/diag/rdbms/tmis/tmis/incident/incdir_41761/tmis_w00c_2224_i41761.trc
2024-07-26T02:15:09.192815+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_m001_31846.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
2024-07-26T02:15:09.199230+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_w00n_2416.trc  (incident=41849):
ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], []
ORA-27300: OS system dependent operation:semop failed with status: 22
ORA-27301: OS failure message: Invalid argument
ORA-27302: failure occurred at: sskgpwwait2b
ORA-27303: additional information: ctx(0x7f4a6ba61000); wid(0xae4c7550); flags(200)
semid(0x458004); sem_flg(0x0); sem_num(115); oldval(-1); errno(22)
ORA-00448: normal completion of background process
Incident details in: /oracle/app/oracle/diag/rdbms/tmis/tmis/incident/incdir_41849/tmis_w00n_2416_i41849.trc
2024-07-26T02:15:09.244841+08:00
opidrv aborting process CL02 ospid (32100) as a result of ORA-27157
2024-07-26T02:15:09.246377+08:00
Errors in file /oracle/app/oracle/diag/rdbms/tmis/tmis/trace/tmis_cl02_32100.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1

2、异常分析
查询资料确认到很可能与定时任务脚本有关系，定时任务如下.
[oracle@rhel7 ~]$ crontab -l
15 2 * * * /home/oracle/oel/scripts/rmarch.sh &>/dev/null
20 2 * * * find /home/oracle/oel/logs/* -name "rmarch_*.log" -mtime +7 -exec rm -f {} \;
25 2 * * * find /home/oracle/oel/logs/* -name "backupdb_*.log" -mtime +7 -exec rm -f {} \;
30 2 * * * /home/oracle/oel/scripts/backupdb.sh &>/dev/null

说明：异常开始时间其实是从凌晨2:15开始出现，与定时任务时间吻合.确实也发现手动执行定时任务涉及的脚本不会出现异常，但自动执行时就会发生宕机异常.

3、解决方案
--在/etc/systemd/logind.conf修改RemoveIPC值
# vi /etc/systemd/logind.conf
RemoveIPC=no

--重启服务
# systemctl daemon-reload
# systemctl restart systemd-logind

参考网址：https://www.modb.pro/db/210276