ORA-27603: Cell storage I/O error

最新推荐文章于 2024-07-31 08:30:00 发布

原创最新推荐文章于 2024-07-31 08:30:00 发布 · 2.9k 阅读

0 ·

CC 4.0 BY-SA版权

Exadata 专栏收录该内容

13 篇文章

订阅专栏

博客详细记录了遇到ORA-27603 Cell存储I/O错误的过程，包括错误日志和trc文件的内容。尽管ASM自动恢复了错误，但建议收集sundiag输出、opatch lsinventory信息以及/var/log/messages进行进一步分析。根据文档，这种单个I/O错误是可恢复的，并且不应立即更换磁盘，除非触发预测性故障或关键Cell警告。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

检查alert日志发现如下报错：

Sat Jan 09 04:12:44 2016
Errors in file /u01/PROD/db/tech_st/11.2.0/admin/PROD_erpdb/diag/rdbms/prod/PROD1/trace/PROD1_ora_45452.trc:
ORA-27603: Cell storage I/O error, I/O failed on disk o/192.168.10.3/DATA_ERP_CD_02_erpcel01 at offset 14628683776 for data length 1048576
ORA-27626: Exadata error: 201 (Generic I/O error)
WARNING: Read Failed. group:1 disk:8 AU:3487 offset:3145728 size:1048576
path:o/192.168.10.3/DATA_ERP_CD_02_erpcel01
incarnation:0xe96869b1 asynchronous result:'I/O error'
subsys:OSS krq:0x7f28746c68f8 bufp:0x7f2870906000 osderr1:0xc9 osderr2:0x0
WARNING: failed to read mirror side 1 of virtual extent 502 logical extent 0 of file 533 in group [1.3407387008] from disk DATA_ERP_CD_02_ERPCEL01 allocation unit 3487 reason error; if possible, will try another mirror side
NOTE: successfully read mirror side 2 of virtual extent 502 logical extent 1 of file 533 in group [1.3407387008] from disk DATA_ERP_CD_04_ERPCEL02 allocation unit 11020
Sat Jan 09 04:19:02 2016

以上错误在alert中只出现了一次

去看下PROD1_ora_45452.trc：

*** 2016-01-09 04:12:44.873
*** SESSION ID:(6545.547) 2016-01-09 04:12:44.873
*** CLIENT ID:() 2016-01-09 04:12:44.873
*** SERVICE NAME:(SYS$USERS) 2016-01-09 04:12:44.873
*** MODULE NAME:(backup incr datafile) 2016-01-09 04:12:44.873
*** ACTION NAME:(0000304 STARTED16) 2016-01-09 04:12:44.873

ORA-27603: Cell storage I/O error, I/O failed on disk o/192.168.10.3/DATA_ERP_CD_02_erpcel01 at offset 14628683776 for data length 1048576
ORA-27626: Exadata error: 201 (Generic I/O error)
Exadata error:'Generic I/O error'
IO elapsed time: 2762447 usec Time waited on I/O: 1664139 usec
OSSIPC:SKGXP:[cf14160.0]{0}: (45452 <- 13389)SKGXPDOAINVALCON: connection 0xcf17bc0 admno 0x26b05351 scoono 0x227a911 acconn 0x22ebd5f3 getting closed. inactive: 0

由上可见：

参考文档

Exadata: Alert log flooded with ORA-27626 and ORA-27603 errors (文档 ID 1921269.1)

This is a single IO error and ASM was able to recovery it automatically. To analyze the issue further, please provide following information:

+ sundiag output from erpcel01
+ 'opatch lsinventory output' from both GI and RDBMS home
+ /var/log/messages from erpdb01

there is no disk failure on cell side. Please review following:

Determining when Disks should be replaced on Oracle Exadata Database Machine ( Doc ID 1452325.1 )
==> Case N3. ASM logs on the DB node show I/O error messages in *.trc files similar to:
"
These single errors are recoverable using the built-in redundancy of ASM so ASM will initiate re-write of the block that had the error, using the mirrored copy which allows the disk to re-allocate data around any bad blocks in the physical disk media. The disk should not be replaced until the failures are such that they trigger predictive failure or critical cell alerts.
"

You can ignore this error and no further action needed.