【kingsql分享】解决RAC大量UNKNOWN和CRS-0216: Could not stop resource

最新推荐文章于 2024-07-08 08:00:00 发布

转载最新推荐文章于 2024-07-08 08:00:00 发布 · 352 阅读

文章标签：

RAC环境基于不同的系统，集群件是不一样的，例如听说9i的数据库有基于service guard的HP双机，有基于HA的AIX双机，那些都是比较悠久的架构，
遇到那种集群件问题，需要整理卷组和集群件，总之还是比较麻烦的。

10g基于Linux系统的Oracle官方集群件，还是比较强大的，即使出故障，也很快能解决，其实这不是工程师的技术高了，而是集群件的性能强了

1.RAC环境变成这样时，实例均关闭了
[oracle@rac1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE UNKNOWN rac1
ora....C1.lsnr application ONLINE UNKNOWN rac1
ora.rac1.gsd application ONLINE UNKNOWN rac1
ora.rac1.ons application ONLINE UNKNOWN rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application OFFLINE UNKNOWN rac2
ora....C2.lsnr application OFFLINE UNKNOWN rac2
ora.rac2.gsd application ONLINE UNKNOWN rac2
ora.rac2.ons application ONLINE UNKNOWN rac2
ora.rac2.vip application ONLINE ONLINE rac2
ora.testora.db application ONLINE UNKNOWN rac2
ora....ataf.cs application ONLINE UNKNOWN rac2
ora....ra1.srv application ONLINE UNKNOWN rac1
ora....ra2.srv application ONLINE UNKNOWN rac2
ora....a1.inst application ONLINE OFFLINE
ora....a2.inst application ONLINE OFFLINE

2.起初打算重启各个节点的服务
[root@rac1 ~]# /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
[root@rac1 ~]# /etc/init.d/init.crs start
Startup will be queued to init within 90 seconds.

[root@rac2 ~]# /etc/init.d/init.crs stop
Shutting down Oracle Cluster Ready Services (CRS):
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
[root@rac2 ~]# /etc/init.d/init.crs start
Startup will be queued to init within 90 seconds.

3.重启之后查看发现还是如此

[oracle@rac1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE UNKNOWN rac1
ora....C1.lsnr application ONLINE UNKNOWN rac1
ora.rac1.gsd application ONLINE UNKNOWN rac1
ora.rac1.ons application ONLINE UNKNOWN rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application OFFLINE UNKNOWN rac2
ora....C2.lsnr application OFFLINE UNKNOWN rac2
ora.rac2.gsd application ONLINE UNKNOWN rac2
ora.rac2.ons application ONLINE UNKNOWN rac2
ora.rac2.vip application ONLINE ONLINE rac2
ora.testora.db application ONLINE UNKNOWN rac2
ora....ataf.cs application ONLINE UNKNOWN rac2
ora....ra1.srv application ONLINE UNKNOWN rac1
ora....ra2.srv application ONLINE UNKNOWN rac2
ora....a1.inst application ONLINE OFFLINE
ora....a2.inst application ONLINE OFFLINE

4.于是想到，对单个UNKNOWN的服务进行重启，但却失败
[oracle@rac1 ~]$ crs_stop ora.rac1.ASM1.asm
Attempting to stop `ora.rac1.ASM1.asm` on member `rac1`
`ora.rac1.ASM1.asm` on member `rac1` has experienced an unrecoverable failure.
Human intervention required to resume its availability.
CRS-0216: Could not stop resource 'ora.rac1.ASM1.asm'.

5.后来只能尝试对其进行强制重启！成功

[oracle@rac1 ~]$ crs_stop -f ora.rac1.ASM1.asm
Attempting to stop `ora.rac1.ASM1.asm` on member `rac1`
Stop of `ora.rac1.ASM1.asm` on member `rac1` succeeded.
Target set to OFFLINE for `ora.testora.testora1.inst`
[oracle@rac1 ~]$ crs_start -f ora.rac1.ASM1.asm
Attempting to start `ora.rac1.ASM1.asm` on member `rac1`
Start of `ora.rac1.ASM1.asm` on member `rac1` succeeded.

6.再看忽然发现原来好多UNKNOWN的状态已经变成ONLINE

[oracle@rac1 ~]$ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE rac1
ora....C1.lsnr application ONLINE UNKNOWN rac1
ora.rac1.gsd application ONLINE UNKNOWN rac1
ora.rac1.ons application ONLINE UNKNOWN rac1
ora.rac1.vip application ONLINE ONLINE rac1
ora....SM2.asm application ONLINE ONLINE rac2
ora....C2.lsnr application ONLINE ONLINE rac2
ora.rac2.gsd application ONLINE ONLINE rac2
ora.rac2.ons application ONLINE ONLINE rac2
ora.rac2.vip application ONLINE ONLINE rac2
ora.testora.db application ONLINE ONLINE rac1
ora....ataf.cs application ONLINE ONLINE rac2
ora....ra1.srv application ONLINE ONLINE rac1
ora....ra2.srv application ONLINE ONLINE rac2
ora....a1.inst application ONLINE ONLINE rac1
ora....a2.inst application ONLINE ONLINE rac2

7.谁还UNKNOWN，就强制重启谁
[oracle@rac1 ~]$ crs_stop -f ora.rac1.LISTENER_RAC1.lsnr
Attempting to stop `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1`
Stop of `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` succeeded.
[oracle@rac1 ~]$ crs_stop -f ora.rac1.gsd
Attempting to stop `ora.rac1.gsd` on member `rac1`
Stop of `ora.rac1.gsd` on member `rac1` succeeded.
[oracle@rac1 ~]$ crs_stop -f ora.rac1.ons
Attempting to stop `ora.rac1.ons` on member `rac1`
Stop of `ora.rac1.ons` on member `rac1` succeeded.

[oracle@rac1 ~]$ crs_start -f ora.rac1.LISTENER_RAC1.lsnr
Attempting to start `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1`
Start of `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` succeeded.
[oracle@rac1 ~]$ crs_start -f ora.rac1.gsd
Attempting to start `ora.rac1.gsd` on member `rac1`
Start of `ora.rac1.gsd` on member `rac1` succeeded.
[oracle@rac1 ~]$ crs_start -f ora.rac1.ons
Attempting to start `ora.rac1.ons` on member `rac1`
Start of `ora.rac1.ons` on member `rac1` succeeded.

8.最后全都好了！