通过案例学调优之--RECORDS_PER_BLOCK参数

本文详细介绍了RECORDS_PER_BLOCK参数的作用及其对位图索引的影响,并通过实际案例展示了如何通过调整此参数来优化数据库性能。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

通过案例学调优之--RECORDS_PER_BLOCK参数

     RECORDS_PER_BLOCK参数用于设定每个BLOCK中记录数的最大值,其先找到当前表所有BLOCK中容纳的最大行数,并会把这个数字记录到数据字典,以后任何导致BLOCK行数超过这个数字的插入都会被拒绝。

    RECORDS_PER_BLOCK参数是为位图索引而生的,能够改善位图索引的存储,减小位图索引的长度。这样,利用该位图索引的时候,就能获得比较好的效率了。

    测试案例:

1、表默认的存储分析

15:45:46 SCOTT@ prod >create table t3 (x int,y int);
Table created.

15:46:03 SCOTT@ prod >insert into t3 values (1,1);
1 row created.

15:46:12 SCOTT@ prod >insert into t3 values (2,1);
1 row created.

15:46:27 SCOTT@ prod >commit;
Commit complete.

15:48:01 SCOTT@ prod >insert into t3 select rownum+2,1 from all_objects where rownum <=254;
254 rows created.

15:48:37 SCOTT@ prod >create index t3_indx on t3(x);
Index created.

15:48:57 SCOTT@ prod >exec dbms_stats.gather_table_stats(user,'T3',cascade=>true);
PL/SQL procedure successfully completed.

15:49:54 SCOTT@ prod >select count(distinct dbms_rowid.rowid_block_number(rowid)) from t3;
COUNT(DISTINCTDBMS_ROWID.ROWID_BLOCK_NUMBER(ROWID))
---------------------------------------------------
                                                  1

15:53:09 SCOTT@ prod >col segment_name for a20
15:53:21 SCOTT@ prod >select segment_name,EXTENTS,BLOCKS,BYTES from user_segments where segment_name='T3';
SEGMENT_NAME            EXTENTS     BLOCKS      BYTES
-------------------- ---------- ---------- ----------
T3                            1          8      65536

默认值,T3表中的数据存储在一个数据块上。

2、通过RECORDS_PER_BLOCK参数分散数据块的存储

15:57:47 SCOTT@ prod >drop table t3 purge;
Table dropped.
15:59:59 SCOTT@ prod >create table t3 (x int,y int);
Table created.

16:00:08 SCOTT@ prod >insert into t3 values (1,1);
1 row created.

16:00:16 SCOTT@ prod >insert into t3 values (2,1);
1 row created.

16:00:25 SCOTT@ prod >commit;
Commit complete.

16:00:37 SCOTT@ prod >alter table t3 minimize records_per_block;
Table altered.

16:00:54 SCOTT@ prod >insert into t3 select rownum+2,1 from all_objects where rownum <=254;
254 rows created.

16:01:09 SCOTT@ prod >commit;
Commit complete.

17:15:14 SCOTT@ prod >create index t3_indx on t3(x);

Index created.

16:01:12 SCOTT@ prod >exec dbms_stats.gather_table_stats(user,'T3');
PL/SQL procedure successfully completed.

16:01:58 SCOTT@ prod >select count(distinct dbms_rowid.rowid_block_number(rowid)) from t3;
COUNT(DISTINCTDBMS_ROWID.ROWID_BLOCK_NUMBER(ROWID))
---------------------------------------------------
                                                128

16:21:29 SCOTT@ prod >select dbms_rowid.rowid_block_number(rowid),count(0)   from t3
   group by dbms_rowid.rowid_block_number(rowid);

DBMS_ROWID.ROWID_BLOCK_NUMBER(ROWID)   COUNT(0)
------------------------------------ ----------
                                 198          2
                                 138          2
                                 151          2
                                 153          2
                                 167          2
                                 244          2
                                 245          2
                                 247          2
                                 537          2
                                 544          2
                                 134          2
                                 194          2
                                 207          2
                                 147          2
                                 209          2
                                 213          2
                                 155          2
......
                              

128 rows selected.

可以看出,T3表占用了128个数据块!

        测试发现:执行alter table test minimize records_per_block;之后,目前BLOCK中的记录数(的最大值)会应用到以后的新增数据中,也就是,当以后再往表中INSERT数据时,每个BLOCK中可以包含的记录数将与设定records_per_block之前的最大值保持一致。

     需要注意的是:

  • 不能对空表设定此参数。

  • 每个BLOCK中可以包含的记录数的最低下限是2

  • 不能在已经有 bitmap 的表中使用records_per_block参数,也就是说,如果要使用records_per_block参数,必须先alter table xxx minimize records_per_block,然后才能在表上建立索引。

         如果字段的类型、大小、个数发生了改变,那么就会导致一个比较差的结果,这就说明了,这项功能只在于使用在静态的环境中,比如数据仓库。


主要用途:

  • 通过减少同一个block中的记录数,使记录分布于更多的数据块中,可以优化等待块类型为data blockBuffer Busy Wait事件。

  • 其主要用途是提高BITMAP INDEX的存储性能



3、对table访问分析

15:44:39 SYS@ prod >alter system flush buffer_cache;
System altered.

16:07:01 SYS@ prod >show parameter mult
NAME                                 TYPE                             VALUE
------------------------------------ -------------------------------- ------------------------------
db_file_multiblock_read_count        integer                          28
parallel_adaptive_multi_user         boolean                          TRUE

17:32:42 SCOTT@ prod >col object_name for a20
17:32:49 SCOTT@ prod >select object_name,object_id from user_objects where object_name='T3';

OBJECT_NAME           OBJECT_ID
-------------------- ----------
T3                        76505

16:22:19 SCOTT@ prod >alter session set db_file_multiblock_read_count=64;

Session altered.
将数据块以间隔的方式读入内存
16:09:03 SCOTT@ prod >declare
16:09:20   2  num number;
16:09:25   3  begin
16:09:29   4  for i in 1..64
16:09:34   5  loop
16:09:37   6  select y into num from t3 where x=i*4;
16:09:42   7  end loop;
16:09:48   8  end;
16:09:50   9  /
PL/SQL procedure successfully completed.

17:25:29 SYS@ prod >select file#,block#,status,objd from v$bh where file#=4;

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        521 free            76505
         4        521 free            76505
         4        521 free            76505
         4        165 free            76505
         4        165 free            76505
         4        165 free            76505
         4        542 free            76505
         4        542 free            76505
         4        542 free            76505
         4        131 free            76505
         4        131 free            76505
         4        131 free            76505
         4        131 xcur            76505
         4        529 free            76505
         4        529 free            76505
         4        529 free            76505
         4        529 xcur            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        550 free            76505
         4        550 free            76505
         4        550 free            76505
         4        139 free            76505
         4        139 free            76505
         4        139 free            76505
         4        139 xcur            76505
         4        537 free            76505
         4        537 free            76505
         4        537 free            76505
         4          3 free       4294967295
         4          3 free       4294967295
         4        147 free            76505
         4        147 free            76505
         4        147 free            76505
         4        524 free            76505
         4        524 free            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        524 free            76505
         4        545 free            76505
         4        545 free            76505
         4        545 free            76505
         4        545 xcur            76505
         4        134 free            76505
         4        134 free            76505
         4        134 free            76505
         4        134 xcur            76505
         4        155 free            76505
         4        155 free            76505
         4        155 free            76505
         4        155 xcur            76505
         4        532 free            76505
         4        532 free            76505
         4        532 free            76505
         4        532 xcur            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        553 free            76506
         4        142 free            76505
         4        142 free            76505
         4        142 free            76505
         4        163 free            76505
         4        163 free            76505
         4        163 free            76505
         4        540 free            76505
         4        540 free            76505
         4        540 free            76505
         4        129 free            76505
         4        129 free            76505
         4        129 free            76505
         4        150 free            76505
         4        150 free            76505
         4        150 free            76505
         4        150 xcur            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        527 free            76505
         4        527 free            76505
         4        527 free            76505
         4        527 xcur            76505
         4        548 free            76505
         4        548 free            76505
         4        548 free            76505
         4        137 free            76505
         4        137 free            76505
         4        158 free            76505
         4        158 free            76505
         4        535 free            76505
         4        535 free            76505
         4        145 free            76505
         4        145 free            76505
         4        145 xcur            76505
         4        522 free            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        522 free            76505
         4        522 xcur            76505
         4        166 free            76505
         4        166 free            76505
         4        166 xcur            76505
         4        543 free            76505
         4        543 free            76505
         4        543 xcur            76505
         4        132 free            76505
         4        132 free            76505
         4        153 free            76505
         4        153 free            76505
         4        530 free            76505
         4        530 free            76505
         4        551 free            76505
         4        551 free            76505
         4        551 xcur            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        140 free            76505
         4        140 free            76505
         4        161 free            76505
         4        161 free            76505
         4        161 xcur            76505
         4        538 free            76505
         4        538 free            76505
         4        538 xcur            76505
         4        148 free            76505
         4        148 free            76505
         4        148 xcur            76505
         4        525 free            76505
         4        525 free            76505
         4        525 xcur            76505
         4        546 free            76505
         4        546 free            76505
         4        135 free            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        135 free            76505
         4        156 free            76505
         4        156 free            76505
         4        533 free            76505
         4        533 free            76505
         4        554 free            76506
         4        143 free            76505
         4        143 free            76505
         4        143 xcur            76505
         4        164 free            76505
         4        164 free            76505
         4        164 xcur            76505
         4        541 free            76505
         4        541 free            76505
         4        541 xcur            76505
         4        130 free            76505
         4        130 free            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        151 free            76505
         4        151 free            76505
         4        549 free            76505
         4        549 free            76505
         4        549 xcur            76505
         4        138 free            76505
         4        138 free            76505
         4        138 xcur            76505
         4        159 free            76505
         4        159 free            76505
         4        159 xcur            76505
         4          2 free       4294967295
         4        146 free            76505
         4        146 free            76505
         4        523 free            76505
         4        523 free            76505
         4        523 xcur            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        167 free            76505
         4        167 free            76505
         4        544 free            76505
         4        544 free            76505
         4        133 free            76505
         4        133 free            76505
         4        154 free            76505
         4        154 free            76505
         4        154 xcur            76505
         4        531 free            76505
         4        531 free            76505
         4        552 free            76506
         4        141 free            76505
         4        141 free            76505
         4        141 xcur            76505
         4        162 free            76505
         4        162 free            76505

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        539 free            76505
         4        539 free            76505
         4        539 xcur            76505
         4        149 free            76505
         4        149 free            76505
         4        526 free            76505
         4        526 free            76505
         4        547 free            76505
         4        547 free            76505
         4        547 xcur            76505
         4        157 free            76505
         4        157 free            76505
         4        157 xcur            76505
         4        534 free            76505
         4        534 free            76505
         4        534 xcur            76505
         4        555 free            76506

     FILE#     BLOCK# STATUS           OBJD
---------- ---------- ---------- ----------
         4        555 xcur            76506
188 rows selected.
16:14:20 SYS@ prod >grant alter session to scott;
Grant succeeded.

16:14:39 SYS@ prod >conn scott/tiger
Connected.
16:14:42 SCOTT@ prod >alter session set events '10046 trace name context forever,level 12';
Session altered.

16:15:31 SCOTT@ prod >set autotrace trace
16:15:37 SCOTT@ prod >select * from t3 ;
256 rows selected.
Elapsed: 00:00:00.02
Execution Plan
----------------------------------------------------------
Plan hash value: 4161002650
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |   256 |  1792 |    68   (0)| 00:00:01 |
|   1 |  TABLE ACCESS FULL| T3   |   256 |  1792 |    68   (0)| 00:00:01 |
--------------------------------------------------------------------------
Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
        196  consistent gets
          0  physical reads
          0  redo size
       4829  bytes sent via SQL*Net to client
        606  bytes received via SQL*Net from client
         19  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        256  rows processed
        
[oracle@RH6 ~]$ ls -lt /u01/app/oracle/diag/rdbms/prod/prod/trace/|more
total 12056
-rw-r----- 1 oracle oinstall  51244 Nov 19 17:28 prod_ora_3681.trc
-rw-r----- 1 oracle oinstall    199 Nov 19 17:28 prod_ora_3681.trm
-rw-r--r-- 1 oracle oinstall 430401 Nov 19 17:22 alert_prod.log
-rw-r----- 1 oracle oinstall   8230 Nov 19 17:18 prod_ora_3629.trc

[oracle@RH6 ~]$ grep sequen /u01/app/oracle/diag/rdbms/prod/prod/trace/prod_ora_3681.trc
WAIT #10: nam='db file sequential read' ela= 13 file#=4 block#=130 blocks=1 obj#=76505 tim=1416389324098217
WAIT #10: nam='db file sequential read' ela= 7 file#=4 block#=135 blocks=1 obj#=76505 tim=1416389324098716
WAIT #10: nam='db file sequential read' ela= 7 file#=6 block#=193 blocks=1 obj#=76505 tim=1416389324098758
WAIT #10: nam='db file sequential read' ela= 0 file#=6 block#=195 blocks=1 obj#=76505 tim=1416389324098837
WAIT #10: nam='db file sequential read' ela= 0 file#=6 block#=197 blocks=1 obj#=76505 tim=1416389324098837
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=199 blocks=1 obj#=76505 tim=1416389324098874
WAIT #10: nam='db file sequential read' ela= 9 file#=4 block#=137 blocks=1 obj#=76505 tim=1416389324098917
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=140 blocks=1 obj#=76505 tim=1416389324099100
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=142 blocks=1 obj#=76505 tim=1416389324099144
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=200 blocks=1 obj#=76505 tim=1416389324099188
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=202 blocks=1 obj#=76505 tim=1416389324099230
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=204 blocks=1 obj#=76505 tim=1416389324099395
WAIT #10: nam='db file sequential read' ela= 7 file#=6 block#=206 blocks=1 obj#=76505 tim=1416389324099439
WAIT #10: nam='db file sequential read' ela= 223 file#=4 block#=149 blocks=1 obj#=76505 tim=1416389324100699
WAIT #10: nam='db file sequential read' ela= 13 file#=4 block#=151 blocks=1 obj#=76505 tim=1416389324100962
WAIT #10: nam='db file sequential read' ela= 9 file#=6 block#=209 blocks=1 obj#=76505 tim=1416389324101019
WAIT #10: nam='db file sequential read' ela= 9 file#=6 block#=211 blocks=1 obj#=76505 tim=1416389324101319
WAIT #10: nam='db file sequential read' ela= 7 file#=6 block#=213 blocks=1 obj#=76505 tim=1416389324101384
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=215 blocks=1 obj#=76505 tim=1416389324101418
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=153 blocks=1 obj#=76505 tim=1416389324101459
WAIT #10: nam='db file sequential read' ela= 10 file#=4 block#=156 blocks=1 obj#=76505 tim=1416389324101664
WAIT #10: nam='db file sequential read' ela= 9 file#=4 block#=158 blocks=1 obj#=76505 tim=1416389324101716
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=216 blocks=1 obj#=76505 tim=1416389324101770
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=218 blocks=1 obj#=76505 tim=1416389324101813
WAIT #10: nam='db file sequential read' ela= 9 file#=6 block#=220 blocks=1 obj#=76505 tim=1416389324101992
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=222 blocks=1 obj#=76505 tim=1416389324102036
WAIT #10: nam='db file sequential read' ela= 9 file#=4 block#=165 blocks=1 obj#=76505 tim=1416389324102276
WAIT #10: nam='db file sequential read' ela= 7 file#=4 block#=167 blocks=1 obj#=76505 tim=1416389324102309
WAIT #10: nam='db file sequential read' ela= 9 file#=6 block#=233 blocks=1 obj#=76505 tim=1416389324102355
WAIT #10: nam='db file sequential read' ela= 32 file#=6 block#=235 blocks=1 obj#=76505 tim=1416389324102705
WAIT #10: nam='db file sequential read' ela= 9 file#=6 block#=237 blocks=1 obj#=76505 tim=1416389324102931
WAIT #10: nam='db file sequential read' ela= 27 file#=6 block#=239 blocks=1 obj#=76505 tim=1416389324103182
WAIT #10: nam='db file sequential read' ela= 10 file#=6 block#=256 blocks=1 obj#=76505 tim=1416389324103344
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=129 blocks=1 obj#=76505 tim=1416389324103389
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=257 blocks=1 obj#=76505 tim=1416389324103423
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=521 blocks=1 obj#=76505 tim=1416389324103466
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=524 blocks=1 obj#=76505 tim=1416389324103678
WAIT #10: nam='db file sequential read' ela= 7 file#=4 block#=526 blocks=1 obj#=76505 tim=1416389324103722
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=240 blocks=1 obj#=76505 tim=1416389324103766
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=242 blocks=1 obj#=76505 tim=1416389324103808
WAIT #10: nam='db file sequential read' ela= 9 file#=6 block#=244 blocks=1 obj#=76505 tim=1416389324103872
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=246 blocks=1 obj#=76505 tim=1416389324103918
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=533 blocks=1 obj#=76505 tim=1416389324104170
WAIT #10: nam='db file sequential read' ela= 7 file#=4 block#=535 blocks=1 obj#=76505 tim=1416389324104206
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=249 blocks=1 obj#=76505 tim=1416389324104250
WAIT #10: nam='db file sequential read' ela= 9 file#=6 block#=251 blocks=1 obj#=76505 tim=1416389324104449
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=253 blocks=1 obj#=76505 tim=1416389324104512
WAIT #10: nam='db file sequential read' ela= 8 file#=6 block#=255 blocks=1 obj#=76505 tim=1416389324104544
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=537 blocks=1 obj#=76505 tim=1416389324104584
WAIT #10: nam='db file sequential read' ela= 9 file#=4 block#=540 blocks=1 obj#=76505 tim=1416389324104759
WAIT #10: nam='db file sequential read' ela= 7 file#=4 block#=542 blocks=1 obj#=76505 tim=1416389324104802
WAIT #10: nam='db file sequential read' ela= 8 file#=4 block#=544 blocks=1 obj#=76505 tim=1416389324104845
WAIT #10: nam='db file sequential read' ela= 76 file#=4 block#=546 blocks=1 obj#=76505 tim=1416389324105604
WAIT #10: nam='db file sequential read' ela= 7 file#=4 block#=548 blocks=1 obj#=76505 tim=1416389324105805
WAIT #10: nam='db file sequential read' ela= 6 file#=4 block#=550 blocks=1 obj#=76505 tim=1416389324105834
......

     以上向我们展示了Oracle多个数据块读取的工作机制,当内存中已经有了某个数据块时,Oracle将不再从磁盘中读取它。这里使用一个循环来通过索引块访问的方式(每次读取一个数据块),将间隔的数据块读入到内存中。这样,当我们对T3表执行全表扫描时,尽管设置了参数:

16:22:19 SCOTT@ prod >alter session set db_file_multiblock_read_count=64;

     但是由于没有连续的数据块可以读取了,所以Oracle每次也只能将一个数据块读取到内存。在等待事件中每一个WAIT#中 blocks=1说明每次I/O读取的数据块都为1,而且数据块的序号正好间隔为1,说明她们之间的那个数据块已经读取到内存中了。因为需要读取的数据块不再连续,所以此时不能一次读取多个数据块。

    多数据块读取一般发生在:

      FTS(FULL TABLE SCAN)

      INDEX_FFS(INDEX FAST FULL SCAN)



本文出自 “天涯客的blog” 博客,请务必保留此出处http://tiany.blog.51cto.com/513694/1579707

报错:15:14:55.528 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/public/home/xiaoshunpeng/miniconda3/envs/chip-seq/share/picard-3.4.0-0/picard.jar!/com/intel/gkl/native/libgkl_compression.so [Sat Jul 26 15:14:55 CST 2025] MarkDuplicates --INPUT /public/home/xiaoshunpeng/chip_samtools/P-T-H-K16-1/paired_aligned_sorted.bam --OUTPUT /public/home/xiaoshunpeng/chip_picard/P-T-H-K16-1/picard-1.bam --METRICS_FILE /public/home/xiaoshunpeng/chip_picard/P-T-H-K16-1/picard-1.metrics --REMOVE_DUPLICATES true --ASSUME_SORTED true --CREATE_INDEX true --MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP 50000 --MAX_FILE_HANDLES_FOR_READ_ENDS_MAP 8000 --SORTING_COLLECTION_SIZE_RATIO 0.25 --TAG_DUPLICATE_SET_MEMBERS false --REMOVE_SEQUENCING_DUPLICATES false --TAGGING_POLICY DontTag --CLEAR_DT true --DUPLEX_UMI false --FLOW_MODE false --FLOW_DUP_STRATEGY FLOW_QUALITY_SUM_STRATEGY --FLOW_USE_END_IN_UNPAIRED_READS false --FLOW_USE_UNPAIRED_CLIPPED_END false --FLOW_UNPAIRED_END_UNCERTAINTY 0 --FLOW_UNPAIRED_START_UNCERTAINTY 0 --FLOW_SKIP_FIRST_N_FLOWS 0 --FLOW_Q_IS_KNOWN_END false --FLOW_EFFECTIVE_QUALITY_THRESHOLD 15 --ADD_PG_TAG_TO_READS true --DUPLICATE_SCORING_STRATEGY SUM_OF_BASE_QUALITIES --PROGRAM_RECORD_ID MarkDuplicates --PROGRAM_GROUP_NAME MarkDuplicates --READ_NAME_REGEX <optimized capture of last three &#39;:&#39; separated fields as numeric values> --OPTICAL_DUPLICATE_PIXEL_DISTANCE 100 --MAX_OPTICAL_DUPLICATE_SET_SIZE 300000 --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 5 --MAX_RECORDS_IN_RAM 500000 --CREATE_MD5_FILE false --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false
最新发布
07-27
<think>我们注意到报错信息中包含了大量的默认参数,这些参数在命令行中并未指定,但是被自动添加了。这可能是由于Picard工具内部设置的默认参数。然而,从报错信息来看,并没有明显的错误,只是显示了运行的参数配置。但是用户可能遇到了程序运行失败或者未达到预期的情况。 但是,根据之前的问题,用户可能是在参数格式上遇到了问题。之前的警告是混合使用了新旧参数格式,现在用户已经统一使用新格式(长选项),但是运行后出现了大量的默认参数输出,这通常是Picard在INFO级别日志中打印的。 如果程序运行成功,则不需要担心;如果运行失败,我们需要查看具体的错误信息。然而,用户提供的日志中没有错误,只有INFO信息。 但是,请注意:在提供的命令中,有一个参数`--ASSUME_SORTED`,在Picard MarkDuplicates中,正确的参数名应该是`--ASSUME_SORT_ORDER`(指定排序顺序,如coordinate, queryname等)或者`--ASSUME_SORTED`(布尔值,表示是否假设已排序,但已过时)。在Picard 3.4.0版本中,推荐使用`--ASSUME_SORT_ORDER`。 因此,我们可能需要将`--ASSUME_SORTED true`改为`--ASSUME_SORT_ORDER coordinate`(假设输入文件是按坐标排序的,这是常见的)。 另外,注意`--REMOVE_DUPLICATES`参数,如果设置为true,则会实际删除重复的reads,而不是仅仅标记。 如果用户希望按照坐标排序的BAM文件进行去重,那么应该使用`--ASSUME_SORT_ORDER coordinate`。 修改后的命令如下:</think>### 解决 Picard MarkDuplicates 日志信息问题 您提供的不是错误信息,而是 Picard 工具的标准运行日志。这些信息表示程序正在正常执行,显示了加载的库和完整的参数配置。以下是关键分析和化建议: #### 1. 日志信息解析 ```plaintext 15:14:55.528 INFO NativeLibraryLoader - Loading libgkl_compression.so ``` - **含义**:Picard 正在加载 Intel Genomics Kernel Library (GKL) 压缩库 - **作用**:加速 BAM 文件的压缩/解压缩操作 - **正常性**:这是标准启动过程,非错误 ```plaintext [Sat Jul 26 15:14:55 CST 2025] MarkDuplicates --INPUT ... --OUTPUT ... ``` - **含义**:显示完整的命令行参数配置 - **作用**:提供执行环境的完整记录,便于- **包含**:所有显式设置的参数 + 默认参数值 #### 2. 参数配置建议 当前命令: ```bash picard MarkDuplicates \ --INPUT /path/to/input.bam \ --OUTPUT /path/to/output.bam \ --METRICS_FILE /path/to/metrics.txt \ --REMOVE_DUPLICATES true \ --ASSUME_SORTED true \ --CREATE_INDEX true ``` 化建议: ```bash picard MarkDuplicates \ --INPUT /path/to/input.bam \ --OUTPUT /path/to/output.bam \ --METRICS_FILE /path/to/metrics.txt \ --REMOVE_DUPLICATES true \ --ASSUME_SORT_ORDER coordinate \ # 更精确的排序假设 --CREATE_INDEX true \ --VALIDATION_STRINGENCY LENIENT \ # 处理格式不规范数据 --TAGGING_POLICY All \ # 标记所有重复reads --OPTICAL_DUPLICATE_PIXEL_DISTANCE 100 # 光重复检测 ``` #### 3. 关键参数解释 | 参数 | 当前值 | 推荐值 | 说明 | |------|--------|--------|------| | `ASSUME_SORTED` | true | `--ASSUME_SORT_ORDER coordinate` | 更精确指定排序类型 | | `VALIDATION_STRINGENCY` | (默认STRICT) | LENIENT | 宽松处理格式问题 | | `TAGGING_POLICY` | (默认DontTag) | All | 标记重复reads | | `MAX_RECORDS_IN_RAM` | 500000 | 根据内存整 | 控制内存使用 | | `COMPRESSION_LEVEL` | 5 | 1 (最快) 或 9 (最小) | 压缩级别平衡 | #### 4. 内存化配置 ```bash # 示例:针对大文件的化配置 picard -Xmx8G MarkDuplicates \ # 分配8GB内存 --INPUT large_file.bam \ --OUTPUT deduped.bam \ --METRICS_FILE metrics.txt \ --ASSUME_SORT_ORDER coordinate \ --CREATE_INDEX true \ --MAX_RECORDS_IN_RAM 1000000 \ # 增加内存记录数 --OPTICAL_DUPLICATE_PIXEL_DISTANCE 100 \ --TAGGING_POLICY OpticalOnly \ # 只标记光重复 --COMPRESSION_LEVEL 1 # 最快压缩 ``` #### 5. 完整执行监控脚本 ```bash #!/bin/bash # picard_monitor.sh INPUT_BAM="/public/home/.../paired_aligned_sorted.bam" OUTPUT_BAM="/public/home/.../picard-1.bam" METRICS_FILE="/public/home/.../picard-1.metrics" LOG_FILE="/public/home/.../picard_log.txt" # 启动时间记录 start_time=$(date +%s) # 执行Picard并记录资源使用 /usr/bin/time -v -o time.log picard MarkDuplicates \ --INPUT "$INPUT_BAM" \ --OUTPUT "$OUTPUT_BAM" \ --METRICS_FILE "$METRICS_FILE" \ --REMOVE_DUPLICATES true \ --ASSUME_SORT_ORDER coordinate \ --CREATE_INDEX true \ --VALIDATION_STRINGENCY LENIENT \ --TAGGING_POLICY All \ 2>&1 | tee "$LOG_FILE" # 结束时间记录 end_time=$(date +%s) # 生成报告 echo "===== PROCESSING REPORT =====" echo "Start time: $(date -d @$start_time)" echo "End time: $(date -d @$end_time)" echo "Duration: $((end_time - start_time)) seconds" # 添加资源使用报告 cat time.log rm time.log # 验证输出 echo -e "\n===== OUTPUT VERIFICATION =====" ls -lh "$OUTPUT_BAM" samtools quickcheck "$OUTPUT_BAM" && echo "BAM file is valid" ``` ### 常见问题解答 **Q: 为什么显示这么多参数?** A: Picard 默认打印完整的参数配置,包括: - 用户显式设置的参数 - 所有默认参数- 内部参数(如 `READ_NAME_REGEX`) **Q: 如何减少日志输出?** ```bash # 添加日志控制参数 picard MarkDuplicates \ ... \ --VERBOSITY ERROR \ # 只显示错误 --QUIET true # 完全静默模式 ``` **Q: 如何确认程序成功执行?** 1. 检查退出代码:`echo $?` (0表示成功) 2. 验证输出文件存在且非空 3. 检查日志结尾是否有成功信息
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值