TSM备份Oracle报错(rman-06012,ANR1401W)

部署运行你感兴趣的模型镜像

问题:

Oracle归档日志备份成功,但2条channel有一个报错。
这里写图片描述

环境:

TSM服务器:2个Oracle node,4个DB2 node,共享8个drive。其中,两个oracle node分别共享其中4个drive。
报错的oracle node为RFIDP。

tsm: TSM>q path
Session established with server TSM: Linux/s390x
  Server Version 6, Release 2, Level 4.0
  Server date/time: 09/28/2015 14:26:33  Last access: 09/28/2015 13:06:41


Source Name     Source Type     Destination     Destination     On-Line
                                Name            Type            
-----------     -----------     -----------     -----------     -------
TSM             SERVER          TS3500LIB1      LIBRARY         Yes    
TSM             SERVER          DRIVER10        DRIVE           Yes    
TSM             SERVER          DRIVER11        DRIVE           Yes    
TSM             SERVER          DRIVER12        DRIVE           Yes    
TSM             SERVER          DRIVER13        DRIVE           Yes    
TSM             SERVER          DRIVER14        DRIVE           Yes    
TSM             SERVER          DRIVER15        DRIVE           Yes    
TSM             SERVER          DRIVER16        DRIVE           Yes    
TSM             SERVER          DRIVER17        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER14        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER15        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER16        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER17        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER14        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER15        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER16        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER17        DRIVE           Yes    

排查过程:

检查TSM actlog

q actlog search=rfidp begint=11:00

这里写图片描述
从日志中发现, 有磁带mount失败。

进一步检查磁带mount失败的原因:

q actlog search=nj0174 begint=11:00

这里写图片描述
从日志中看到,driver14和driver16都无法mount这个磁带。

那么问题是在磁带上还是driver上呢?

首先检查改磁带状态:

tsm: TSM>q vol nj0174 f=d

                   Volume Name: NJ0174
             Storage Pool Name: ORACLE_POOL
             Device Class Name: 3592DEVC
            Estimated Capacity: 12.0 T
       Scaled Capacity Applied: 100
                      Pct Util: 22.9
                 Volume Status: Filling
                        Access: Read/Write
        Pct. Reclaimable Space: 0.0
               Scratch Volume?: Yes
               In Error State?: No
      Number of Writable Sides: 1
       Number of Times Mounted: 138
             Write Pass Number: 1
     Approx. Date Last Written: 09/28/2015 13:50:38
        Approx. Date Last Read: 08/10/2015 01:10:30
           Date Became Pending: 
        Number of Write Errors: 0
         Number of Read Errors: 0
               Volume Location: 
Volume is MVS Lanfree Capable : No
Last Update by (administrator): 
         Last Update Date/Time: 08/10/2015 01:07:44
          Begin Reclaim Period: 
            End Reclaim Period: 
  Drive Encryption Key Manager: None
       Logical Block Protected: No


tsm: TSM>

从上面的输出可以看到磁带状态正常。那么问题有可能在drive上。

继续查看actlog,关于driver14和driver16的记录:

tsm: TSM>q actlog search=driver14 begind=-1

这里写图片描述

tsm: TSM>q actlog search=driver16 begind=-1

这里写图片描述

这两个drive一直都无法mount磁带。问题可以确定就出在这两个drive上。

检查操作系统,带库所有drive状态。(drive过多,省略了若干)
发现driver14正常,driver16有一条路径是offline。

[root@TSM /]# lstape
FICON/ESCON tapes (found 0):
TapeNo  BusID      CuType/Model DevType/Model   BlkSize State   Op      MedState
.
.
.                  
sg54    IBMtape13     31           13:0:1:1 IBM              03592E07 tapedrv
running                                                               
sg55    IBMtape14     32           14:0:0:1 IBM              03592E07 tapedrv
running                                                               
sg56    IBMchanger1   3            14:0:1:1 IBM              03584L22 changer
running                                                               
sg57    IBMtape15     33           14:0:2:1 IBM              03592E07 tapedrv
running                                                               
sg58    IBMtape16     34           15:0:0:1 IBM              03592E07 tapedrv
running                                                               
sg59    IBMtape17     35           15:0:1:1 IBM              03592E07 tapedrv
running                                                               
.
.
.                 
sg35    IBMtape14     32           8:0:0:1  IBM              03592E07 tapedrv
running                                                               
sg36    IBMchanger1   3            8:0:1:1  IBM              03584L22 changer
running                                                               
sg37    IBMtape15     33           8:0:2:1  IBM              03592E07 tapedrv
running                                                               
sg38    N/A           9:0:0:1      IBM      03592E07         tapedrv  offline
sg39    IBMtape17     35           9:0:1:1  IBM              03592E07 tapedrv
running                                                               
[root@TSM /]# 

登陆带库web控制台检查drive状态:
这里写图片描述
两个红框中的drive就是driver14和driver16。表面上看,状态正常。

对这两个drive做一次power cycle试试。
这里写图片描述

这里写图片描述

power cycle成功,在TSM server操作系统中,再次查看drive设备状态:driver16之前offline的路径已经变成running状态。
这里写图片描述

测试:

rman备份发起2个channel,该数据库节点在TSM server中配置了4条path,手动offline 2条,这样可以测试driver14和driver16的path。

tsm: TSM>q path
Session established with server TSM: Linux/s390x
  Server Version 6, Release 2, Level 4.0
  Server date/time: 09/28/2015 14:26:33  Last access: 09/28/2015 13:06:41


Source Name     Source Type     Destination     Destination     On-Line
                                Name            Type            
-----------     -----------     -----------     -----------     -------
.
.
.
RFIDPSTA1       SERVER          DRIVER14        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER15        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER16        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER17        DRIVE           Yes  

手动offline driver15和driver17


update path RFIDPSTA1 driver15 srctype=server  desttype=drive DEVICE=/dev/IBMtape1 LIBRARY=ts3500lib1 online=no
update path RFIDPSTA1 driver17 srctype=server  desttype=drive DEVICE=/dev/IBMtape3 LIBRARY=ts3500lib1 online=no


tsm: TSM>update path RFIDPSTA1 driver15 srctype=server  desttype=drive DEVICE=/dev/IBMtape1 LIBRARY=ts3500lib1 online=no
ANR1726E A operation using the path from RFIDPSTA1 to TS3500LIB1 DRIVER15 is currently being used.
ANS8001I Return code 12.

driver15 in use,需要手动dismount磁带出来再进行offline

tsm: TSM>q mo
ANR8329I 3592 volume NJ0100 is mounted R/W in drive DRIVER10 (/dev/IBMtape10), status: IDLE.
ANR8329I 3592 volume NJ0080 is mounted R/W in drive DRIVER17 (/dev/IBMtape17), status: IDLE.
ANR8329I 3592 volume NJ0054 is mounted R/W in drive DRIVER13 (/dev/IBMtape13), status: IDLE.
ANR8329I 3592 volume NJ0034 is mounted R/W in drive DRIVER12 (/dev/IBMtape12), status: IDLE.
ANR8329I 3592 volume NJ0035 is mounted R/W in drive DRIVER15 (/dev/IBMtape15), status: IDLE.
ANR8334I         5 matches found.

tsm: TSM>dismount vol nj0035
ANR8499I Command accepted.

手动dismount磁带成功。

再次查看mount状态:

tsm: TSM>q mo
ANR8329I 3592 volume NJ0100 is mounted R/W in drive DRIVER10 (/dev/IBMtape10), status: IDLE.
ANR8329I 3592 volume NJ0080 is mounted R/W in drive DRIVER17 (/dev/IBMtape17), status: IDLE.
ANR8329I 3592 volume NJ0054 is mounted R/W in drive DRIVER13 (/dev/IBMtape13), status: IDLE.
ANR8329I 3592 volume NJ0034 is mounted R/W in drive DRIVER12 (/dev/IBMtape12), status: IDLE.
ANR8331I 3592 volume NJ0035 is mounted R/W in drive DRIVER15 (/dev/IBMtape15), status: DISMOUNTING.
ANR8334I         5 matches found.

dismount中

tsm: TSM>q mo
ANR8329I 3592 volume NJ0100 is mounted R/W in drive DRIVER10 (/dev/IBMtape10), status: IDLE.
ANR8329I 3592 volume NJ0054 is mounted R/W in drive DRIVER13 (/dev/IBMtape13), status: IDLE.
ANR8329I 3592 volume NJ0034 is mounted R/W in drive DRIVER12 (/dev/IBMtape12), status: IDLE.
ANR8334I         4 matches found.

mount列表中已无driver15

再次手动offline driver15

tsm: TSM>update path RFIDPSTA1 driver15 srctype=server  desttype=drive DEVICE=/dev/IBMtape1 LIBRARY=ts3500lib1 online=no
ANR1722I A path from RFIDPSTA1 to TS3500LIB1 DRIVER15 has been updated.

offline成功

tsm: TSM>q path

Source Name     Source Type     Destination     Destination     On-Line
                                Name            Type            
-----------     -----------     -----------     -----------     -------
TSM             SERVER          TS3500LIB1      LIBRARY         Yes    
TSM             SERVER          DRIVER10        DRIVE           Yes    
TSM             SERVER          DRIVER11        DRIVE           Yes    
TSM             SERVER          DRIVER12        DRIVE           Yes    
TSM             SERVER          DRIVER13        DRIVE           Yes    
TSM             SERVER          DRIVER14        DRIVE           Yes    
TSM             SERVER          DRIVER15        DRIVE           Yes    
TSM             SERVER          DRIVER16        DRIVE           Yes    
TSM             SERVER          DRIVER17        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER14        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER15        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER16        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER17        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER14        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER15        DRIVE           No    
RFIDPSTA1       SERVER          DRIVER16        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER17        DRIVE           No    

重新运行归档日志备份脚本,测试driver14和driver16是否可用。

查看磁带mount情况,可以看到driver14和driver16已成功mount磁带,并且状态为R/W in use,说明备份正在写入磁带中。可以确定这两个drive已经可以正常使用了。

tsm: TSM>
tsm: TSM>q mo
ANR8329I 3592 volume NJ0100 is mounted R/W in drive DRIVER10 (/dev/IBMtape10), status: IDLE.
ANR8329I 3592 volume NJ0080 is mounted R/W in drive DRIVER13 (/dev/IBMtape13), status: IDLE.
ANR8330I 3592 volume NJ0035 is mounted R/W in drive DRIVER16 (/dev/IBMtape16), status: IN USE.
ANR8329I 3592 volume NJ0034 is mounted R/W in drive DRIVER12 (/dev/IBMtape12), status: IDLE.
ANR8330I 3592 volume NJ0174 is mounted R/W in drive DRIVER14 (/dev/IBMtape14), status: IN USE.
ANR8334I         6 matches found.

继续等待,直到脚本执行完毕,检查脚本日志。备份成功,无报错。

[oracle@RFIDP log]$ tail  /oracle/rmanscript/log/arch_092813.log
archive log filename=/oradata/arch/1_166187_777424034.dbf recid=166178 stamp=891610878
archive log filename=/oradata/arch/1_166188_777424034.dbf recid=166179 stamp=891611321
archive log filename=/oradata/arch/1_166189_777424034.dbf recid=166180 stamp=891611495
Finished backup at 28-SEP-15

released channel: t1

released channel: t2

Recovery Manager complete.

最后不要忘记把刚才offline的path重新online。

tsm: TSM>update path RFIDPSTA1 driver15 srctype=server  desttype=drive DEVICE=/dev/IBMtape1 LIBRARY=ts3500lib1 online=yes
ANR1722I A path from RFIDPSTA1 to TS3500LIB1 DRIVER15 has been updated.

tsm: TSM>update path RFIDPSTA1 driver17 srctype=server  desttype=drive DEVICE=/dev/IBMtape3 LIBRARY=ts3500lib1 online=yes
ANR1722I A path from RFIDPSTA1 to TS3500LIB1 DRIVER17 has been updated.

tsm: TSM>


tsm: TSM>q path

Source Name     Source Type     Destination     Destination     On-Line
                                Name            Type            
-----------     -----------     -----------     -----------     -------
TSM             SERVER          TS3500LIB1      LIBRARY         Yes    
TSM             SERVER          DRIVER10        DRIVE           Yes    
TSM             SERVER          DRIVER11        DRIVE           Yes    
TSM             SERVER          DRIVER12        DRIVE           Yes    
TSM             SERVER          DRIVER13        DRIVE           Yes    
TSM             SERVER          DRIVER14        DRIVE           Yes    
TSM             SERVER          DRIVER15        DRIVE           Yes    
TSM             SERVER          DRIVER16        DRIVE           Yes    
TSM             SERVER          DRIVER17        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER14        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER15        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER16        DRIVE           Yes    
JTFXPSTA1       SERVER          DRIVER17        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER14        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER15        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER16        DRIVE           Yes    
RFIDPSTA1       SERVER          DRIVER17        DRIVE           Yes 

您可能感兴趣的与本文相关的镜像

LobeChat

LobeChat

AI应用

LobeChat 是一个开源、高性能的聊天机器人框架。支持语音合成、多模态和可扩展插件系统。支持一键式免费部署私人ChatGPT/LLM 网络应用程序。

评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值