ORA-15196 while allocation reaches 2Tb even compatible.asm is set to 12.1 (Doc ID 1933068.1)

Oracle ASM不支持大于2T的磁盘，此为bug，在12.1版本上修复了

APPLIES TO: Oracle Database - Enterprise Edition - Version 12.1.0.1 and later Information in this document applies to any platform. SYMPTOMS   New diskgroup was created with compatible.asm 12.1.0.0. While restoring to this diskgroup ,the moment file allocation reaches 2Tb ,ASM diskgroup getting corrupted and  dismounting with ORA-15196 error. While diskgroup was created ,  .  Wed Jun 11 14:20:55 2014  SQL> CREATE DISKGROUP O25T_BACKUP EXTERNAL REDUNDANCY DISK  '/dev/vx/rdsk/o25t_backup_asm_dg/3pardata3_271_vol' SIZE 2044723M ,  '/dev/vx/rdsk/o25t_backup_asm_dg/3pardata3_272_vol' SIZE 2044723M ATTRIBUTE  .  'compatible.asm'='12.1.0.0.0','compatible.rdbms'='11.2','au_size'='4M' /*  ASMCA */ >>>>>>>>>>>>>diskgroup created with 2T disks  Wed Jun 11 14:20:55 2014  NOTE: Assigning number (13,0) to disk  (/dev/vx/rdsk/o25t_backup_asm_dg/3pardata3_271_vol)  NOTE: Assigning number (13,1) to disk  (/dev/vx/rdsk/o25t_backup_asm_dg/3pardata3_272_vol)  NOTE: initializing header (replicated) on grp 13 disk O25T_BACKUP_0000  NOTE: initializing header (replicated) on grp 13 disk O25T_BACKUP_0001  + Size was 1.94+ Tb.  .  Thu Jun 12 09:25:51 2014  SUCCESS: ALTER DISKGROUP O25T_BACKUP RESIZE DISK O25T_BACKUP_0000 DISK  O25T_BACKUP_0001 /* EM */  Starting background process ARB0  Thu Jun 12 09:25:51 2014  ARB0 started with pid=60, OS id=45134  NOTE: assigning ARB0 to group 13/0x82998fc7 (O25T_BACKUP) with 1024 parallel  I/Os  NOTE: Attempting voting file refresh on diskgroup O25T_BACKUP  Thu Jun 12 09:27:01 2014  NOTE: requesting all-instance membership refresh for group=13  Thu Jun 12 09:27:01 2014  NOTE: membership refresh pending for group 13/0x82998fc7 (O25T_BACKUP)  Thu Jun 12 09:27:01 2014  GMON querying group 13 at 15604 for pid 15, osid 9205  Thu Jun 12 09:27:01 2014  SUCCESS: refreshed membership for 13/0x82998fc7 (O25T_BACKUP)  NOTE: Attempting voting file refresh on diskgroup O25T_BACKUP  Thu Jun 12 09:28:15 2014  NOTE: stopping process ARB0  Thu Jun 12 09:28:16 2014  SUCCESS: rebalance completed for group 13/0x82998fc7 (O25T_BACKUP)  Thu Jun 12 09:31:58 2014  ERROR: failed to read ACD block gn=13 blk=0  Thu Jun 12 09:31:58 2014  Errors in file /opt/ora/gridOB/diag/asm/+asm/+ASM/trace/+ASM_lgwr_9197.trc:  ORA-15196: invalid ASM block header [kfr.c:10703] [endian_kfbh] [3] [0] [6 !=  1]  WARNING: LGWR failed to read ACDC for diskgroup 13 thread 1  Thu Jun 12 09:31:58 2014  Errors in file /opt/ora/gridOB/diag/asm/+asm/+ASM/trace/+ASM_lgwr_9197.trc:  Thu Jun 12 09:31:58 2014  WARNING: cache read (retry) a corrupt block: group=13(O25T_BACKUP) fn=1 blk=1  disk=0 (O25T_BACKUP_0000) incarn=3916006032 au=10 blk=1 count=1  Thu Jun 12 09:31:58 2014  Errors in file /opt/ora/gridOB/diag/asm/+asm/+ASM/trace/+ASM_ora_19652.trc:  ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  NOTE: a corrupted block from group O25T_BACKUP was dumped to  /opt/ora/gridOB/diag/asm/+asm/+ASM/trace/+ASM_ora_19652.trc  ERROR: cache failed to read group=13(O25T_BACKUP) fn=1 blk=1 from disk(s):  0(O25T_BACKUP_0000)  ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  .  NOTE: cache initiating offline of disk 0 group O25T_BACKUP  NOTE: process _user19652_+asm (19652) initiating offline of disk 0.3916006032  (O25T_BACKUP_0000) with mask 0x7e in group 13 (O25T_BACKUP) with client  assisting  NOTE: initiating PST update: grp 13 (O25T_BACKUP), dsk = 0/0xe9698290, mask =  0x6a, op = clear  Thu Jun 12 09:31:58 2014  GMON updating disk modes for group 13 at 16730 for pid 54, osid 19652  ERROR: disk 0(O25T_BACKUP_0000) in group 13(O25T_BACKUP) cannot be offlined  because the disk group has external redundancy.  Thu Jun 12 09:31:58 2014  ERROR: too many offline disks in PST (grp 13)  Thu Jun 12 09:31:58 2014  NOTE: cache dismounting (not clean) group 13/0x82998FC7 (O25T_BACKUP)  NOTE: messaging CKPT to quiesce pins Unix process pid: 35225, image:  oraclesehan9022cc.han.telia.se (B000)  Thu Jun 12 09:31:58 2014  NOTE: halting all I/Os to diskgroup 13 (O25T_BACKUP)  Thu Jun 12 09:31:58 2014  NOTE: LGWR doing non-clean dismount of group 13 (O25T_BACKUP) thread 1  NOTE: LGWR sync ABA=14.4118 last written ABA 14.4118  Thu Jun 12 09:31:58 2014  NOTE: cache dismounted group 13/0x82998FC7 (O25T_BACKUP)  Thu Jun 12 09:31:58 2014  SQL> alter diskgroup O25T_BACKUP dismount force /* ASM SERVER:2191101895 */  Errors in file /opt/ora/gridOB/diag/asm/+asm/+ASM/trace/+ASM_ora_19652.trc  (incident=176433):  ORA-15335: ASM metadata corruption detected in disk group 'O25T_BACKUP'  ORA-15130: diskgroup "O25T_BACKUP" is being dismounted  ORA-15066: offlining disk "O25T_BACKUP_0000" in group "O25T_BACKUP" may  result in a data loss  ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  Incident details in:  /opt/ora/gridOB/diag/asm/+asm/+ASM/incident/incdir_176433/+ASM_ora_19652_i1764  33.trc  Thu Jun 12 09:31:58 2014  ASM Health Checker found 1 new failures  Thu Jun 12 09:31:58 2014  ASM Health Checker found 1 new failures  Thu Jun 12 09:31:58 2014  ERROR: ORA-15130 in COD recovery for diskgroup 13/0x82998fc7 (O25T_BACKUP)  ERROR: ORA-15130 thrown in RBAL for group number 13  Thu Jun 12 09:31:58 2014  Errors in file /opt/ora/gridOB/diag/asm/+asm/+ASM/trace/+ASM_rbal_9205.trc:  ORA-15130: diskgroup "O25T_BACKUP" is being dismounted  .  .  .  kfbh.endian: 73 ; 0x000: 0x49  kfbh.hard: 0 ; 0x001: 0x00  kfbh.type: 110 ; 0x002: *** Unknown Enum ***  kfbh.datfmt: 0 ; 0x003: 0x00  kfbh.block.blk: 7274614 ; 0x004: blk=7274614  kfbh.block.obj: 6881387 ; 0x008: file=589931  kfbh.check: 6750318 ; 0x00c: 0x0067006e  kfbh.fcn.base: 7929939 ; 0x010: 0x00790053  kfbh.fcn.wrap: 7602291 ; 0x014: 0x00740073  kfbh.spare1: 7143525 ; 0x018: 0x006d0065  kfbh.spare2: 4063266 ; 0x01c: 0x003e0022  kfbtTraverseBlock: Invalid OSM block type 110 ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  ORA-15196: invalid ASM block header [kfc.c:28346] [endian_kfbh] [1] [1] [73  != 1]  ----- Abridged Call Stack Trace -----  ksedsts()+213<-kfcReadBlk()+1824<-kfcLoad()+6326<-kffbScanNext()+279<-kffbTabl  eCb()+861<-kfgTableCb()+986<-kffilTableCb()+203<-qerfxFetch()+3847<-qersoProce  ssULS()+308<-qersoFetch()+6012<-qerjotFetch()+2147<-rwsfcd()+124<-qersoProcess  ULS()+308<-qersoFetchSimple()+1042  <-qersoFetch()+225<-qerjotFetch()+2385<-rwsfcd()+124<-qersoProcessULS()+308<-q  ersoFetchSimple()+1042<-qersoFetch()+225<-qerjotFetch()+2385<-rwsfcd()+124<-qe  rsoProcessULS()+308<-qersoFetchSimple()+1042<-qersoFetch()+225<-qerjotFetch()+  2385<-rwsfcd()+124<-qersoProcessULS()+308  <-qersoFetchSimple()+1042<-qersoFetch()+225<-qerjotFetch()+2385<-qergsFetch()+  531<-rwsfcd()+124<-qeruaFetch()+348<-qervwFetch()+144<-qergsFetch()+531<-rwsfc  d()+124<-qeruaFetch()+348<-qersoProcessULS()+308<-qersoFetchSimple()+1042<-qer  soFetch()+225<-opifch2()+3148  <-kpoal8()+3713<-opiodr()+1170<-ttcpip()+2660<-opitsk()+1751<-opiino()+888<-op  iodr()+1170<-opidrv()+586<-sou2o()+120<-opimai_real()+151<-ssthrdmain()+392<-m  ain()+222<-__libc_start_main()+253  ----- End of Abridged Call Stack Trace -----  NOTE: process _user19652_+asm (19652) initiating offline of disk 0.3916006032  (O25T_BACKUP_0000) with mask 0x7e in group 13 (O25T_BACKUP) with client  assisting  NOTE: initiating PST update: grp 13 (O25T_BACKUP), dsk = 0/0xe9698290, mask =  0x6a, op = clear  kfdp_updateDsk(): 16730  DDE rules only execution for: ORA 15066  .  .  File_name :: +ASM_ora_19652.trc Disk was becoming CANDIDATE ,though no external writes were taking place . CAUSE  without Exadata storage,11g RDBMS software does not support larger than 2TB ASM disk. SOLUTION   + For current corruption ,you need to recreate this diskgroup and restore data from backup. For future to avoid such issue , + Apply fix 19012119 as per read me of patch . + Without Exadata storage, set compatible.asm and compatible.rdbms disk group attribute is set to 12.1 or greater. REFERENCES BUG:19012119 - ORA-15196 WITH 5TB DISK SIZE IN 12.1