本文记录了一次Oracle 11gr2 RAC环境中，一个节点频繁重启的故障。故障表现为VIP漂移、数据库无法连接ASM并报错。通过对主机日志、数据库日志、ASM日志及网络抓包的分析，确定故障原因为RAC节点私有网络丢包。网络专家发现节点间的通信异常，存在丢包现象。最后通过关闭特定网络端口避免ARP请求，解决了丢包问题，从而防止了节点重启。但故障的根本原因未明。
故障现象：
2011年的一次问题，oracle 11gr2 rac + redhat linux ，2节点rac中的其中一个节点频繁重启；
原因分析：
主机日志
VIP发生了漂移，重启后又归位
node1
| Nov 23 18:22:27 dtydb2 avahi-daemon[13096]: Withdrawing address record for 10.4.124.242 on bond2. Nov 23 18:22:31 dtydb2 avahi-daemon[13096]: Withdrawing address record for 169.254.188.250 on bond1. Nov 23 18:23:10 dtydb2 avahi-daemon[13096]: Registering new address record for 169.254.188.250 on bond1. Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Registering new address record for 10.4.124.242 on bond2. Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Withdrawing address record for 10.4.124.242 on bond2. Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Registering new address record for 10.4.124.242 on bond2. Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Withdrawing address record for 10.4.124.242 on bond2. Nov 23 18:23:35 dtydb2 avahi-daemon[13096]: Registering new address record for 10.4.124.242 on bond2. node2 Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Registering new address record for 10.4.124.242 on bond2. Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Withdrawing address record for 10.4.124.242 on bond2. Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Registering new address record for 10.4.124.242 on bond2. Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Withdrawing address record for 10.4.124.242 on bond2. Nov 23 18:22:31 dtydb1 avahi-daemon[13132]: Registering new address record for 10.4.124.242 on bond2. Nov 23 18:23:34 dtydb1 avahi-daemon[13132]: Withdrawing address record for 10.4.124.242 on bond2. |
数据库日志
不能连接ASM，所有重启
ORA-15064: communication failure with ASMinstance
ORA-03113: end-of-file on communicationchannel
ASM日志
最低0.47元/天 解锁文章
扫一扫
304
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
