Oracle RAC备库启动service报"ORA-16000: database open for read-only access"

Oracle RAC备库启动service报"ORA-16000: database open for read-only access"

还是2019.03.01那天的事了，当时在KFT客户就遇到这个问题，最近在规整一些资料看到当时待整理的文档，就抽空做做实验整理下。

报错信息如下，ADG备库：

[oracle@xxxprdoradb01 ~]$ srvctl start service -d xxxdbdr2 -s xxxdbcss
PRCD-1084 : Failed to start service xxxdbcss
PRCR-1079 : Failed to start resource ora.xxxdbdr2.xxxdbcss.svc
CRS-5017: The resource action "ora.xxxdbdr2.xxxdbcss.svc start" encountered the following error: 
ORA-16000: database open for read-only access
ORA-06512: at "SYS.DBMS_SERVICE", line 433
ORA-06512: at "SYS.DBMS_SERVICE", line 304
ORA-06512: at line 1
. For details refer to "(:CLSN00107:)" in "/u01/oracle/grid/110203/log/xxxprdoradb01/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2674: Start of 'ora.xxxdbdr2.xxxdbcss.svc' on 'xxxprdoradb01' failed
CRS-5017: The resource action "ora.xxxdbdr2.xxxdbcss.svc start" encountered the following error: 
ORA-16000: database open for read-only access
ORA-06512: at "SYS.DBMS_SERVICE", line 433
ORA-06512: at "SYS.DBMS_SERVICE", line 304
ORA-06512: at line 1
. For details refer to "(:CLSN00107:)" in "/u01/oracle/grid/110203/log/xxxprdoradb02/agent/crsd/oraagent_oracle/oraagent_oracle.log".

CRS-2674: Start of 'ora.xxxdbdr2.xxxdbcss.svc' on 'xxxprdoradb02' failed
CRS-2632: There are no more servers to try to place resource 'ora.xxxdbdr2.xxxdbcss.svc' on that would satisfy its placement policy

其中，/u01/oracle/grid/110203/log/xxxprdoradb01/agent/crsd/oraagent_oracle/oraagent_oracle.log文件相关内容如下：

2019-03-01 19:55:48.191: [    AGFW][3892311808] {
     1:14336:11864} Agent received the message: AGENT_HB[Engine] ID 12293:2664818
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] ClusterSubscriber::SubscriberWorker::InternalClusterSubscriber::handleEventCBexecuting for reason 1
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] event type is CRS_ACTION_FAILURE
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] bodylen = 538
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597} -----------BodyBlock----------
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  ACTION='1'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  CLS_TINT='{1:14336:56597}'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  CURRENT_STATE='OFFLINE'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  DB_UNIQUE_NAME='xxxdbdr2'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  INSTANCE_NAME='xxxdb2'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  NAME='ora.xxxdbdr2.xxxdbcss.svc'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  NUMBER_OF_ATTEMPTS='0'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  REASON='USER_ACTION'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  RESOURCE_CLASS='service'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  RESOURCE_INCARNATION_NUMBER='0'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  RESOURCE_LOCATION='xxxprdoradb02'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  SEQUENCE_NUMBER='1400583'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  SERVICE_NAME=''
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  TARGET_STATE='ONLINE'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  TIMESTAMP='2019-03-01 19:55:49'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  TYPE='ora.service.type'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  USER='oracle'
2019-03-01 19:55:49.300: [ USRTHRD][3758094080] {
     1:14338:56597}  Version='11.2.0.3.0'
2019-03-01 19:55:49.301: [ USRTHRD][3758094080] {
     1:14338:56597}  CLUSTER_NAME='xxxdb01-scan'
2019-03-01 19:55:49.301: [ USRTHRD][3758094080] {
     1:14338:56597}  DB_UNIQUE_NAME='xxxdbdr2'
2019-03-01 19:55:49.301: [ USRTHRD][3758094080] {
     1:14338:56