开发人员反映数据库连接不稳定,连到节点一经常超时
[grid@rac01 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.FRA.dg
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.LISTENER.lsnr
ONLINE INTERMEDIATE rac01 Not All Endpoints R
egistered
ONLINE ONLINE rac02
ora.asm
ONLINE ONLINE rac01 Started
ONLINE ONLINE rac02 Started
ora.gsd
OFFLINE OFFLINE rac01
OFFLINE OFFLINE rac02
ora.net1.network
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.ons
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.registry.acfs
ONLINE ONLINE rac01
ONLINE ONLINE rac02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac02
ora.LISTENER_SCAN2.lsnr
1 ONLINE INTERMEDIATE rac01 Not All Endpoints R
egistered
ora.LISTENER_SCAN3.lsnr
1 ONLINE INTERMEDIATE rac01 Not All Endpoints R
egistered
ora.cvu
1 ONLINE ONLINE rac01
ora.oc4j
1 ONLINE ONLINE rac01
ora.pacs.db
1 ONLINE ONLINE rac01 Open
2 ONLINE ONLINE rac02 Open
ora.rac01.vip
1 ONLINE ONLINE rac01
ora.rac02.vip
1 ONLINE ONLINE rac02
ora.scan1.vip
1 ONLINE ONLINE rac02
ora.scan2.vip
1 ONLINE ONLINE rac01
ora.scan3.vip
1 ONLINE ONLINE rac01
查看监听的进程,发现oracle用户下也有个一个监听进程,是不应该的
[grid@rac01 ~]$ ps -ef |grep LISTENER
oracle 5455 1 0 Aug06 ? 00:00:38 /u01/app/oracle/product/11.2.0/db_1/bin/tnslsnr LISTENER -inherit
grid 5755 1 0 Aug06 ? 00:00:09 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid 6630 6557 0 17:38 pts/3 00:00:00 grep LISTENER
grid 14320 1 0 09:45 ? 00:00:02 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN2 -inherit
grid 14361 1 0 09:45 ? 00:00:02 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN3 -inherit
直接kill
[root@rac01 ~]# kill -9 5455
过会检查状态:
[grid@rac01 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.FRA.dg
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.LISTENER.lsnr
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.asm
ONLINE ONLINE rac01 Started
ONLINE ONLINE rac02 Started
ora.gsd
OFFLINE OFFLINE rac01
OFFLINE OFFLINE rac02
ora.net1.network
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.ons
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.registry.acfs
ONLINE ONLINE rac01
ONLINE ONLINE rac02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac02
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE rac01
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE rac01
ora.cvu
1 ONLINE ONLINE rac01
ora.oc4j
1 ONLINE ONLINE rac01
ora.pacs.db
1 ONLINE ONLINE rac01 Open
2 ONLINE ONLINE rac02 Open
ora.rac01.vip
1 ONLINE ONLINE rac01
ora.rac02.vip
1 ONLINE ONLINE rac02
ora.scan1.vip
1 ONLINE ONLINE rac02
ora.scan2.vip
1 ONLINE ONLINE rac01
ora.scan3.vip
1 ONLINE ONLINE rac01
4,metailink上也有此报错的文章,主要是listener端口被占用导致不能正确启动导致
Cause
The problem is caused by another listener defined statically in listener.ora, using the same port and IP is running from the RDBMS ORACLE_HOME, started manually causing the default listener starting from GRID_HOME can not register its endpoint. Hence the
error reported in dbca.
ps -ef | grep tns:
grid 7222 1 0 Apr26 ? 00:00:13 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid 7237 1 0 Apr26 ? 00:00:13 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
oracle 7354 1 0 Apr26 ? 00:00:01 /u02/app/oracle/product/11.2.0/db/bin/tnslsnr LISTENER -inherit
Another possible cause is the listener or scan listener being defined manually in listener.ora, for example:
LISTENER_SCAN3 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = racnode1)(PORT = 1523))
)
LISTENER_SCAN1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = racnode1)(PORT = 1521))
Solution
From 11.2 onwards, all listeners should be runing from GRID_HOME, listener and listener_scan<n> entry should be added automatically into listener.ora, no manual editing is required for TCP definition.
1. Stop the listener running from RDBMS ORACLE_HOME
$<RDBMS ORACLE_HOME>/bin/lsnrctl stop LISTENER
2. stop the listener from GRID_HOME
$<GRID_HOME>/bin/srvctl stop listener -n <node name>
$<GRID_HOME>/bin/srvctl stop scan_listener -i <scan#>
eg:
$<GRID_HOME>/bin/srvctl stop listener -n racnode1
$<GRID_HOME>/bin/srvctl stop scan_listener -i 1
If above command fails to stop the tnslsnr process, please use "kill -9 <pid of tnslsnr>" to stop the LISTENER and LISTENER_SCAN1 process.
3. remove any manually added LISTENER definition from listener.ora if it exists
4. restart the LISTENER and LISTENER_SCAN1 from GRID_HOME
$<GRID_HOME>/bin/srvctl start listener -n <node name>
$<GRID_HOME>/bin/srvctl start scan_listener -i <scan#>
5. check crsctl stat res -t output, they both should show ONLINE status now.