工具使用-性能基准测试工具-fio(测试磁盘、文件系统、应用程序的io性能)

最新推荐文章于 2025-02-06 15:15:13 发布

勇敢的星火

最新推荐文章于 2025-02-06 15:15:13 发布

阅读量2.9k

点赞数 16

文章标签：测试工具

本文链接：https://blog.youkuaiyun.com/qinaide56/article/details/139329873

版权

参考来源；

linux 磁盘IO测试工具：fio (同时简要介绍dd工具测试)
https://www.cnblogs.com/xuyaowen/p/fio-usage.html
linux FIO命令详解(一)：磁盘IO测试工具 fio (并简要介绍iostat工具) https://blog.youkuaiyun.com/don_chiang709/article/details/92628623
磁盘测试工具FIO工具安装和使用方法
https://blog.youkuaiyun.com/zhiboqingyun/article/details/123368887
Fio工具详解【强大的IO性能压测工具】
https://cloud.tencent.com/developer/article/2322582

极客时间专栏：Linux性能优化实战、深入浅出云计算
gitcode关于工具的介绍：https://gitcode.com/axboe/fio

工具的介绍和安装

目前主流的第三方IO测试工具有fio、iometer 和 Orion，这三种工具各有千秋，在linux 下也可以使用dd 进行简单的磁盘（文件系统）测试（文末补充）。

fio在Linux系统下使用比较方便，iometer在window系统下使用比较方便，Orion是oracle的IO测试软件，可在没有安装oracle数据库的情况下模拟oracle数据库场景的读写。

Multithreaded IO generation tool , FIO 是一个多线程io生成工具，可以生成多种IO模式，用来测试磁盘设备的性能（也包含文件系统：如针对网络文件系统 NFS 的IO测试）；

fio（Flexible I/O Tester）正是最常用的文件系统和磁盘 I/O 性能基准测试工具。它提供了大量的可定制化选项，可以用来测试，裸盘或者文件系统在各种场景下的 I/O 性能，包括了不同块大小、不同 I/O 引擎以及是否使用缓存等场景。
fio 的安装比较简单，你可以执行下面的命令来安装它：

# Ubuntu
apt-get install -y fio
# CentOS
yum install -y fio

安装完成后，就可以执行 man fio 查询它的使用方法。

常见场景

fio 的选项非常多，我会通过几个常见场景的测试方法，介绍一些最常用的选项。这些常见场景包括随机读、随机写、顺序读以及顺序写等，你可以执行下面这些命令来测试：

# 随机读
fio -name=randread -direct=1 -iodepth=64 -rw=randread -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sdb


# 随机写
fio -name=randwrite -direct=1 -iodepth=64 -rw=randwrite -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sdb


# 顺序读
fio -name=read -direct=1 -iodepth=64 -rw=read -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sdb

# 顺序写
fio -name=write -direct=1 -iodepth=64 -rw=write -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/sdb

在这其中，有几个参数需要你重点关注一下。
direct，表示是否跳过系统缓存。上面示例中，我设置的 1 ，就表示跳过系统缓存。
iodepth，表示使用异步 I/O（asynchronous I/O，简称 AIO）时，同时发出的 I/O 请求上限。在上面的示例中，我设置的是 64。
rw，表示 I/O 模式。我的示例中， read/write 分别表示顺序读 / 写，而 randread/randwrite 则分别表示随机读 / 写。
ioengine，表示 I/O 引擎，它支持同步（sync）、异步（libaio）、内存映射（mmap）、网络（net）等各种 I/O 引擎。上面示例中，我设置的 libaio 表示使用异步 I/O。
bs，表示 I/O 的大小。示例中，我设置成了 4K（这也是默认值）。
filename，表示文件路径，当然，它可以是磁盘路径（测试磁盘性能），也可以是文件路径（测试文件系统性能）。示例中，我把它设置成了磁盘 /dev/sdb。不过注意，用磁盘路径测试写，会破坏这个磁盘中的文件系统，所以在使用前，你一定要事先做好数据备份。

下面就是我使用 fio 测试顺序读的一个报告示例。

root@ub1804:/home/xiaoyue# fio -ioengine=libaio -bs=4k -direct=1 -thread -rw=read -filename=/dev/sda -name="BS 4KB read test" -iodepth=16 -runtime=60  -size=1G
BS 4KB read test: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.1
Starting 1 thread
Jobs: 1 (f=1): [R(1)][100.0%][r=97.4MiB/s,w=0KiB/s][r=24.9k,w=0 IOPS][eta 00m:00s]
BS 4KB read test: (groupid=0, jobs=1): err= 0: pid=5426: Thu May 30 15:29:33 2024
   **read: IOPS=10.1k, BW=39.3MiB/s (41.2MB/s)(1024MiB/26076msec)**
    slat (usec): min=16, max=18762, avg=38.35, stdev=44.63
    clat (nsec): min=1529, max=127409k, avg=1551393.72, stdev=4924855.91
     lat (usec): min=31, max=127451, avg=1590.24, stdev=4927.67
    clat percentiles (usec):
     |  1.00th=[   383],  5.00th=[   433], 10.00th=[   441], 20.00th=[   453],
     | 30.00th=[   482], 40.00th=[   510], 50.00th=[   553], 60.00th=[   676],
     | 70.00th=[   758], 80.00th=[   889], 90.00th=[  1270], 95.00th=[  3261],
     | 99.00th=[ 23987], 99.50th=[ 31851], 99.90th=[ 57410], 99.95th=[ 68682],
     | 99.99th=[126354]
   bw (  KiB/s): min= 3728, max=118704, per=99.37%, **avg=39960.04**, stdev=33568.28, samples=52
   iops        : min=  932, max=29676, **avg=9989.88**, stdev=8392.14, samples=52
  lat (usec)   : 2=0.01%, 4=0.01%, 10=0.01%, 50=0.12%, 100=0.01%
  lat (usec)   : 250=0.38%, 500=35.39%, 750=32.84%, 1000=16.45%
  lat (msec)   : 2=8.03%, 4=1.92%, 10=1.00%, 20=2.30%, 50=1.43%
  lat (msec)   : 100=0.12%, 250=0.02%
  cpu          : usr=0.51%, sys=43.74%, ctx=10370, majf=0, minf=17
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=262144,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=39.3MiB/s (41.2MB/s), 39.3MiB/s-39.3MiB/s (41.2MB/s-41.2MB/s), io=1024MiB (1074MB), run=26076-26076msec

Disk stats (read/write):
  sda: ios=256049/7, merge=712/3, ticks=293034/2, in_queue=205596, util=99.56%
root@ub1804:/home/xiaoyue#

诊断输出内容解读：

可以参考文档链接：https://tobert.github.io/post/2014-04-17-fio-output-explained.html
这个报告中，需要我们重点关注的是， slat、clat、lat ，以及 bw 和 iops 这几行。

先来看刚刚提到的前三个参数。事实上，slat、clat、lat 都是指 I/O 延迟（latency）。不同之处在于：
slat ，是指从 I/O 提交到实际执行 I/O 的时长（Submission latency）；
clat ，是指从 I/O 提交到 I/O 完成的时长（Completion latency）；
而 lat ，指的是从 fio 创建 I/O 到 I/O 完成的总时长。
这里需要注意的是，对同步 I/O 来说，由于 I/O 提交和 I/O 完成是一个动作，所以 slat 实际上就是 I/O 完成的时间，而 clat 是 0。而从示例可以看到，使用异步 I/O（libaio）时，lat 近似等于 slat + clat 之和。

再来看 bw ，它代表吞吐量。在我上面的示例中，你可以看到，平均吞吐量大约是 39 MB（39960KiB/1024）。
最后的 iops ，其实就是**每秒 I/O 的次数**，上面示例中的平均 IOPS 为 9989。

应用程序的 I/O模式的基准测试

通常情况下，应用程序的 I/O 都是读写并行的，而且每次的 I/O 大小也不一定相同。所以，刚刚说的这几种场景，并不能精确模拟应用程序的 I/O 模式。那怎么才能精确模拟应用程序的 I/O 模式呢？

幸运的是，fio 支持 I/O 的重放。借助前面提到过的 blktrace，再配合上 fio，就可以实现对应用程序 I/O 模式的基准测试。你需要先用 blktrace ，记录磁盘设备的 I/O 访问情况；然后使用 fio ，重放 blktrace 的记录。

比如你可以运行下面的命令来操作：

# 使用 blktrace 跟踪磁盘 I/O，**注意指定应用程序正在操作的磁盘**
$ blktrace /dev/sdb
# 查看 blktrace 记录的结果
# ls
sdb.blktrace.0 sdb.blktrace.1
# 将结果转化为二进制文件
$ blkparse sdb -d sdb.bin
# 使用 fio 重放日志
$ fio --name=replay --filename=/dev/sdb --direct=1 --read_iolog=sdb.bin

这样，我们就通过 blktrace+fio 的组合使用，得到了应用程序 I/O 模式的基准测试报告

blktrace输出示例

root@ub1804:/home/xiaoyue# blktrace /dev/sdb
^C=== sdb ===
  CPU  0:                  307 events,       15 KiB data
  CPU  1:                   10 events,        1 KiB data
  CPU  2:                    0 events,        0 KiB data
  CPU  3:                    0 events,        0 KiB data
  Total:                   317 events (dropped 0),       15 KiB data

blkparse将结果转化为二进制文件

root@ub1804:/home/xiaoyue# blkparse sdb -d sdb.bin
Input file sdb.blktrace.0 added
Input file sdb.blktrace.1 added
  8,16   0        1     0.000000000  4690  Q  RA 0 + 8 [fdisk]
  8,16   0        2     0.000002951  4690  G  RA 0 + 8 [fdisk]
  8,16   0        3     0.000003324  4690  P   N [fdisk]
  8,16   0        4     0.000004097  4690  U   N [fdisk] 1
  8,16   0        5     0.000004734  4690  I  RA 0 + 8 [fdisk]
  8,16   0        6     0.000007116  4690  D  RA 0 + 8 [fdisk]
  8,16   0        7     0.000105121  4690  C  RA 0 + 8 [0]
  8,16   0        8     0.000146417  4690  Q  RA 8 + 8 [fdisk]
  8,16   0        9     0.000147954  4690  G  RA 8 + 8 [fdisk]
  8,16   0       10     0.000148310  4690  P   N [fdisk]
  8,16   0       11     0.000148578  4690  U   N [fdisk] 1
  8,16   0       12     0.000149218  4690  I  RA 8 + 8 [fdisk]
  8,16   0       13     0.000150392  4690  D  RA 8 + 8 [fdisk]
  8,16   0       14     0.000184787  4690  C  RA 8 + 8 [0]
  8,16   0       15    21.921933043  4691  Q  RA 10485632 + 8 [fdisk]
  8,16   0       16    21.921937364  4691  G  RA 10485632 + 8 [fdisk]
  8,16   0       17    21.921937912  4691  P   N [fdisk]
  8,16   0       18    21.921938406  4691  U   N [fdisk] 1
  8,16   0       19    21.921939115  4691  I  RA 10485632 + 8 [fdisk]
  8,16   0       20    21.921942471  4691  D  RA 10485632 + 8 [fdisk]
  8,16   0       21    21.922018828  4691  C  RA 10485632 + 8 [0]
  8,16   0       22    21.922032896  4691  Q  RA 10485744 + 8 [fdisk]
  8,16   0       23    21.922048001  4691  G  RA 10485744 + 8 [fdisk]
  8,16   0       24    21.922048307  4691  P   N [fdisk]
  8,16   0       25    21.922048512  4691  U   N [fdisk] 1
  8,16   0       26    21.922048815  4691  I  RA 10485744 + 8 [fdisk]
  8,16   0       27    21.922049695  4691  D  RA 10485744 + 8 [fdisk]
  8,16   0       28    21.922105089  4691  C  RA 10485744 + 8 [0]
  8,16   0       29    21.922123039  4691  Q  RA 10485752 + 8 [fdisk]
  8,16   0       30    21.922123930  4691  G  RA 10485752 + 8 [fdisk]
  8,16   0       31    21.922124169  4691  P   N [fdisk]
  8,16   0       32    21.922124352  4691  U   N [fdisk] 1
  8,16   0       33    21.922124693  4691  I  RA 10485752 + 8 [fdisk]
  8,16   0       34    21.922125584  4691  D  RA 10485752 + 8 [fdisk]
  8,16   0       35    21.922152076  4691  C  RA 10485752 + 8 [0]
  8,16   0       36    21.922162253  4691  Q  RA 10485496 + 8 [fdisk]
  8,16   0       37    21.922163124  4691  G  RA 10485496 + 8 [fdisk]
  8,16   0       38    21.922163331  4691  P   N [fdisk]
  8,16   0       39    21.922163514  4691  U   N [fdisk] 1
  8,16   0       40    21.922163820  4691  I  RA 10485496 + 8 [fdisk]
  8,16   0       41    21.922164605  4691  D  RA 10485496 + 8 [fdisk]
  8,16   0       42    21.922202903  4691  C  RA 10485496 + 8 [0]
  8,16   0       43    21.922216176  4691  Q  RA 10485696 + 8 [fdisk]
  8,16   0       44    21.922217016  4691  G  RA 10485696 + 8 [fdisk]
  8,16   0       45    21.922217215  4691  P   N [fdisk]
  8,16   0       46    21.922217393  4691  U   N [fdisk] 1
  8,16   0       47    21.922217688  4691  I  RA 10485696 + 8 [fdisk]
  8,16   0       48    21.922218496  4691  D  RA 10485696 + 8 [fdisk]
  8,16   0       49    21.922241366  4691  C  RA 10485696 + 8 [0]
  8,16   0       50    21.922274396  4691  Q  RA 10485504 + 8 [fdisk]
  8,16   0       51    21.922275266  4691  G  RA 10485504 + 8 [fdisk]
  8,16   0       52    21.922275502  4691  P   N [fdisk]
  8,16   0       53    21.922275683  4691  U   N [fdisk] 1
  8,16   0       54    21.922276005  4691  I  RA 10485504 + 8 [fdisk]
  8,16   0       55    21.922276785  4691  D  RA 10485504 + 8 [fdisk]
  8,16   0       56    21.922349894  4691  C  RA 10485504 + 8 [0]
  8,16   0       57    21.922375316  4691  Q  RA 10485360 + 8 [fdisk]
  8,16   0       58    21.922376072  4691  G  RA 10485360 + 8 [fdisk]
  8,16   0       59    21.922376289  4691  P   N [fdisk]
  8,16   0       60    21.922376468  4691  U   N [fdisk] 1
  8,16   0       61    21.922376775  4691  I  RA 10485360 + 8 [fdisk]
  8,16   0       62    21.922377463  4691  D  RA 10485360 + 8 [fdisk]
  8,16   0       63    21.922425796  4691  C  RA 10485360 + 8 [0]
  8,16   0       64    21.922436341  4691  Q  RA 10485168 + 8 [fdisk]
  8,16   0       65    21.922437156  4691  G  RA 10485168 + 8 [fdisk]
  8,16   0       66    21.922437390  4691  P   N [fdisk]
  8,16   0       67    21.922437588  4691  U   N [fdisk] 1
  8,16   0       68    21.922437925  4691  I  RA 10485168 + 8 [fdisk]
  8,16   0       69    21.922438633  4691  D  RA 10485168 + 8 [fdisk]
  8,16   0       70    21.922463814  4691  C  RA 10485168 + 8 [0]
  8,16   0       71    21.922473192  4691  Q  RA 10485080 + 8 [fdisk]
  8,16   0       72    21.922474020  4691  G  RA 10485080 + 8 [fdisk]
  8,16   0       73    21.922474229  4691  P   N [fdisk]
  8,16   0       74    21.922474409  4691  U   N [fdisk] 1
  8,16   0       75    21.922474736  4691  I  RA 10485080 + 8 [fdisk]
  8,16   0       76    21.922475444  4691  D  RA 10485080 + 8 [fdisk]
  8,16   0       77    21.922512948  4691  C  RA 10485080 + 8 [0]
  8,16   0       78    21.922521262  4691  Q  RA 10485024 + 8 [fdisk]
  8,16   0       79    21.922522043  4691  G  RA 10485024 + 8 [fdisk]
  8,16   0       80    21.922522247  4691  P   N [fdisk]
  8,16   0       81    21.922522403  4691  U   N [fdisk] 1
  8,16   0       82    21.922522781  4691  I  RA 10485024 + 8 [fdisk]
  8,16   0       83    21.922523413  4691  D  RA 10485024 + 8 [fdisk]
  8,16   0       84    21.922545167  4691  C  RA 10485024 + 8 [0]
  8,16   0       85    21.922553676  4691  Q  RA 10484848 + 8 [fdisk]
  8,16   0       86    21.922554466  4691  G  RA 10484848 + 8 [fdisk]
  8,16   0       87    21.922554676  4691  P   N [fdisk]
  8,16   0       88    21.922554863  4691  U   N [fdisk] 1
  8,16   0       89    21.922555177  4691  I  RA 10484848 + 8 [fdisk]
  8,16   0       90    21.922555831  4691  D  RA 10484848 + 8 [fdisk]
  8,16   0       91    21.922614725  4691  C  RA 10484848 + 8 [0]
  8,16   0       92    21.922624486  4691  Q  RA 10484784 + 8 [fdisk]
  8,16   0       93    21.922625349  4691  G  RA 10484784 + 8 [fdisk]
  8,16   0       94    21.922625581  4691  P   N [fdisk]
  8,16   0       95    21.922625755  4691  U   N [fdisk] 1
  8,16   0       96    21.922626095  4691  I  RA 10484784 + 8 [fdisk]
  8,16   0       97    21.922626831  4691  D  RA 10484784 + 8 [fdisk]
  8,16   0       98    21.922665086  4691  C  RA 10484784 + 8 [0]
  8,16   0       99    21.922673341  4691  Q  RA 10484768 + 8 [fdisk]
  8,16   0      100    21.922674088  4691  G  RA 10484768 + 8 [fdisk]
  8,16   0      101    21.922674286  4691  P   N [fdisk]
  8,16   0      102    21.922674461  4691  U   N [fdisk] 1
  8,16   0      103    21.922674773  4691  I  RA 10484768 + 8 [fdisk]
  8,16   0      104    21.922675415  4691  D  RA 10484768 + 8 [fdisk]
  8,16   0      105    21.922711072  4691  C  RA 10484768 + 8 [0]
  8,16   0      106    21.922718722  4691  Q  RA 10484808 + 8 [fdisk]
  8,16   0      107    21.922719387  4691  G  RA 10484808 + 8 [fdisk]
  8,16   0      108    21.922719561  4691  P   N [fdisk]
  8,16   0      109    21.922719700  4691  U   N [fdisk] 1
  8,16   0      110    21.922719967  4691  I  RA 10484808 + 8 [fdisk]
  8,16   0      111    21.922720541  4691  D  RA 10484808 + 8 [fdisk]
  8,16   0      112    21.922740573  4691  C  RA 10484808 + 8 [0]
  8,16   0      113    21.922763180  4691  Q  RA 10482672 + 8 [fdisk]
  8,16   0      114    21.922764071  4691  G  RA 10482672 + 8 [fdisk]
  8,16   0      115    21.922764258  4691  P   N [fdisk]
  8,16   0      116    21.922764410  4691  U   N [fdisk] 1
  8,16   0      117    21.922764685  4691  I  RA 10482672 + 8 [fdisk]
  8,16   0      118    21.922765338  4691  D  RA 10482672 + 8 [fdisk]
  8,16   0      119    21.922787538  4691  C  RA 10482672 + 8 [0]
  8,16   0      120    21.922815240  4691  Q  RA 2048 + 8 [fdisk]
  8,16   0      121    21.922816103  4691  G  RA 2048 + 8 [fdisk]
  8,16   0      122    21.922816296  4691  P   N [fdisk]
  8,16   0      123    21.922816484  4691  U   N [fdisk] 1
  8,16   0      124    21.922816796  4691  I  RA 2048 + 8 [fdisk]
  8,16   0      125    21.922817601  4691  D  RA 2048 + 8 [fdisk]
  8,16   0      126    21.922854632  4691  C  RA 2048 + 8 [0]
  8,16   0      127    21.922869744  4691  Q  RA 24 + 8 [fdisk]
  8,16   0      128    21.922870483  4691  G  RA 24 + 8 [fdisk]
  8,16   0      129    21.922870652  4691  P   N [fdisk]
  8,16   0      130    21.922870792  4691  U   N [fdisk] 1
  8,16   0      131    21.922871057  4691  I  RA 24 + 8 [fdisk]
  8,16   0      132    21.922871667  4691  D  RA 24 + 8 [fdisk]
  8,16   0      133    21.922891939  4691  C  RA 24 + 8 [0]
  8,16   0      134    21.922901482  4691  Q  RA 56 + 8 [fdisk]
  8,16   0      135    21.922902190  4691  G  RA 56 + 8 [fdisk]
  8,16   0      136    21.922902365  4691  P   N [fdisk]
  8,16   0      137    21.922902524  4691  U   N [fdisk] 1
  8,16   0      138    21.922902798  4691  I  RA 56 + 8 [fdisk]
  8,16   0      139    21.922903380  4691  D  RA 56 + 8 [fdisk]
  8,16   0      140    21.922923296  4691  C  RA 56 + 8 [0]
  8,16   0      141    21.922931429  4691  Q  RA 120 + 8 [fdisk]
  8,16   0      142    21.922932134  4691  G  RA 120 + 8 [fdisk]
  8,16   0      143    21.922932307  4691  P   N [fdisk]
  8,16   0      144    21.922932435  4691  U   N [fdisk] 1
  8,16   0      145    21.922932689  4691  I  RA 120 + 8 [fdisk]
  8,16   0      146    21.922933265  4691  D  RA 120 + 8 [fdisk]
  8,16   0      147    21.922953082  4691  C  RA 120 + 8 [0]
  8,16   0      148    21.922971354  4691  Q  RA 16 + 8 [fdisk]
  8,16   0      149    21.922972176  4691  G  RA 16 + 8 [fdisk]
  8,16   0      150    21.922972353  4691  P   N [fdisk]
  8,16   0      151    21.922972508  4691  U   N [fdisk] 1
  8,16   0      152    21.922972790  4691  I  RA 16 + 8 [fdisk]
  8,16   0      153    21.922973548  4691  D  RA 16 + 8 [fdisk]
  8,16   0      154    21.923012454  4691  C  RA 16 + 8 [0]
  8,16   0      155    21.923018525  4691  Q  RA 32 + 24 [fdisk]
  8,16   0      156    21.923019237  4691  G  RA 32 + 24 [fdisk]
  8,16   0      157    21.923019412  4691  P   N [fdisk]
  8,16   0      158    21.923019535  4691  U   N [fdisk] 1
  8,16   0      159    21.923019790  4691  I  RA 32 + 24 [fdisk]
  8,16   0      160    21.923020420  4691  D  RA 32 + 24 [fdisk]
  8,16   0      161    21.923046159  4691  C  RA 32 + 24 [0]
  8,16   0      162    21.923057045  4691  Q  RA 64 + 56 [fdisk]
  8,16   0      163    21.923057794  4691  G  RA 64 + 56 [fdisk]
  8,16   0      164    21.923057957  4691  P   N [fdisk]
  8,16   0      165    21.923058092  4691  U   N [fdisk] 1
  8,16   0      166    21.923058368  4691  I  RA 64 + 56 [fdisk]
  8,16   0      167    21.923058972  4691  D  RA 64 + 56 [fdisk]
  8,16   0      168    21.923089034  4691  C  RA 64 + 56 [0]
  8,16   0      169    21.923130025  4691  Q  RA 128 + 384 [fdisk]
  8,16   0      170    21.923131102  4691  G  RA 128 + 384 [fdisk]
  8,16   0      171    21.923131275  4691  P   N [fdisk]
  8,16   0      172    21.923131416  4691  U   N [fdisk] 1
  8,16   0      173    21.923131715  4691  I  RA 128 + 384 [fdisk]
  8,16   0      174    21.923132345  4691  D  RA 128 + 384 [fdisk]
  8,16   0      175    21.923202518  4691  C  RA 128 + 384 [0]
  8,16   0      176    21.923361618  4691  Q  RA 512 + 512 [fdisk]
  8,16   0      177    21.923363160  4691  G  RA 512 + 512 [fdisk]
  8,16   0      178    21.923363361  4691  P   N [fdisk]
  8,16   0      179    21.923363525  4691  U   N [fdisk] 1
  8,16   0      180    21.923363844  4691  I  RA 512 + 512 [fdisk]
  8,16   0      181    21.923364581  4691  D  RA 512 + 512 [fdisk]
  8,16   0      182    21.923489526  4691  C  RA 512 + 512 [0]
  8,16   0      183    21.923731426  4691  Q  RA 10484736 + 32 [fdisk]
  8,16   0      184    21.923733002  4691  G  RA 10484736 + 32 [fdisk]
  8,16   0      185    21.923733251  4691  P   N [fdisk]
  8,16   0      186    21.923733666  4691  U   N [fdisk] 1
  8,16   0      187    21.923734045  4691  I  RA 10484736 + 32 [fdisk]
  8,16   0      188    21.923735253  4691  D  RA 10484736 + 32 [fdisk]
  8,16   0      189    21.923785483  4691  C  RA 10484736 + 32 [0]
  8,16   0      190    21.923791545  4691  Q  RA 10484776 + 8 [fdisk]
  8,16   0      191    21.923792430  4691  G  RA 10484776 + 8 [fdisk]
  8,16   0      192    21.923792608  4691  P   N [fdisk]
  8,16   0      193    21.923792762  4691  U   N [fdisk] 1
  8,16   0      194    21.923793057  4691  I  RA 10484776 + 8 [fdisk]
  8,16   0      195    21.923793791  4691  D  RA 10484776 + 8 [fdisk]
  8,16   0      196    21.923819678  4691  C  RA 10484776 + 8 [0]
  8,16   0      197    21.923825278  4691  Q  RA 10484792 + 16 [fdisk]
  8,16   0      198    21.923826113  4691  G  RA 10484792 + 16 [fdisk]
  8,16   0      199    21.923826305  4691  P   N [fdisk]
  8,16   0      200    21.923826515  4691  U   N [fdisk] 1
  8,16   0      201    21.923826799  4691  I  RA 10484792 + 16 [fdisk]
  8,16   0      202    21.923827488  4691  D  RA 10484792 + 16 [fdisk]
  8,16   0      203    21.923867599  4691  C  RA 10484792 + 16 [0]
  8,16   0      204    21.923897022  4691  Q  RA 10484816 + 32 [fdisk]
  8,16   0      205    21.923897971  4691  G  RA 10484816 + 32 [fdisk]
  8,16   0      206    21.923898195  4691  P   N [fdisk]
  8,16   0      207    21.923898434  4691  U   N [fdisk] 1
  8,16   0      208    21.923898762  4691  I  RA 10484816 + 32 [fdisk]
  8,16   0      209    21.923899525  4691  D  RA 10484816 + 32 [fdisk]
  8,16   0      210    21.923940726  4691  C  RA 10484816 + 32 [0]
  8,16   0      211    21.924023039  4691  Q  RA 10484856 + 168 [fdisk]
  8,16   0      212    21.924024416  4691  G  RA 10484856 + 168 [fdisk]
  8,16   0      213    21.924024655  4691  P   N [fdisk]
  8,16   0      214    21.924024884  4691  U   N [fdisk] 1
  8,16   0      215    21.924025201  4691  I  RA 10484856 + 168 [fdisk]
  8,16   0      216    21.924025990  4691  D  RA 10484856 + 168 [fdisk]
  8,16   0      217    21.924053578  4691  C  RA 10484856 + 168 [0]
  8,16   0      218    21.924080053  4691  Q  RA 10485032 + 48 [fdisk]
  8,16   0      219    21.924080816  4691  G  RA 10485032 + 48 [fdisk]
  8,16   0      220    21.924080999  4691  P   N [fdisk]
  8,16   0      221    21.924081140  4691  U   N [fdisk] 1
  8,16   0      222    21.924081389  4691  I  RA 10485032 + 48 [fdisk]
  8,16   0      223    21.924137160  4691  D  RA 10485032 + 48 [fdisk]
  8,16   0      224    21.924180341  4691  C  RA 10485032 + 48 [0]
  8,16   0      225    21.924195472  4691  Q  RA 10485088 + 80 [fdisk]
  8,16   0      226    21.924196689  4691  G  RA 10485088 + 80 [fdisk]
  8,16   0      227    21.924196938  4691  P   N [fdisk]
  8,16   0      228    21.924197131  4691  U   N [fdisk] 1
  8,16   0      229    21.924197431  4691  I  RA 10485088 + 80 [fdisk]
  8,16   0      230    21.924198204  4691  D  RA 10485088 + 80 [fdisk]
  8,16   0      231    21.924222565  4691  C  RA 10485088 + 80 [0]
  8,16   0      232    21.924265692  4691  Q  RA 10485176 + 72 [fdisk]
  8,16   0      233    21.924266737  4691  G  RA 10485176 + 72 [fdisk]
  8,16   0      234    21.924266932  4691  P   N [fdisk]
  8,16   0      235    21.924267184  4691  U   N [fdisk] 1
  8,16   0      236    21.924267441  4691  I  RA 10485176 + 72 [fdisk]
  8,16   0      237    21.924268172  4691  D  RA 10485176 + 72 [fdisk]
  8,16   0      238    21.924289529  4691  C  RA 10485176 + 72 [0]
  8,16   0      239    21.924510235  4691  Q  RA 10485248 + 112 [fdisk]
  8,16   0      240    21.924511288  4691  G  RA 10485248 + 112 [fdisk]
  8,16   0      241    21.924511525  4691  P   N [fdisk]
  8,16   0      242    21.924511696  4691  U   N [fdisk] 1
  8,16   0      243    21.924511996  4691  I  RA 10485248 + 112 [fdisk]
  8,16   0      244    21.924512867  4691  D  RA 10485248 + 112 [fdisk]
  8,16   0      245    21.924557489  4691  C  RA 10485248 + 112 [0]
  8,16   0      246    21.924593993  4691  Q  RA 10485368 + 128 [fdisk]
  8,16   0      247    21.924594888  4691  G  RA 10485368 + 128 [fdisk]
  8,16   0      248    21.924595115  4691  P   N [fdisk]
  8,16   0      249    21.924595499  4691  U   N [fdisk] 1
  8,16   0      250    21.924595803  4691  I  RA 10485368 + 128 [fdisk]
  8,16   0      251    21.924596562  4691  D  RA 10485368 + 128 [fdisk]
  8,16   0      252    21.924621723  4691  C  RA 10485368 + 128 [0]
  8,16   0      253    21.924636726  4691  Q  RA 10485512 + 120 [fdisk]
  8,16   0      254    21.924637619  4691  G  RA 10485512 + 120 [fdisk]
  8,16   0      255    21.924637829  4691  P   N [fdisk]
  8,16   0      256    21.924637996  4691  U   N [fdisk] 1
  8,16   0      257    21.924638276  4691  I  RA 10485512 + 120 [fdisk]
  8,16   0      258    21.924638928  4691  D  RA 10485512 + 120 [fdisk]
  8,16   0      259    21.924698375  4691  C  RA 10485512 + 120 [0]
  8,16   0      260    21.924707825  4691  Q  RA 10485640 + 56 [fdisk]
  8,16   0      261    21.924708618  4691  G  RA 10485640 + 56 [fdisk]
  8,16   0      262    21.924708831  4691  P   N [fdisk]
  8,16   0      263    21.924708983  4691  U   N [fdisk] 1
  8,16   0      264    21.924709261  4691  I  RA 10485640 + 56 [fdisk]
  8,16   0      265    21.924709922  4691  D  RA 10485640 + 56 [fdisk]
  8,16   0      266    21.924733762  4691  C  RA 10485640 + 56 [0]
  8,16   0      267    21.924741732  4691  Q  RA 10485704 + 40 [fdisk]
  8,16   0      268    21.924742539  4691  G  RA 10485704 + 40 [fdisk]
  8,16   0      269    21.924742732  4691  P   N [fdisk]
  8,16   0      270    21.924742897  4691  U   N [fdisk] 1
  8,16   0      271    21.924743193  4691  I  RA 10485704 + 40 [fdisk]
  8,16   0      272    21.924743897  4691  D  RA 10485704 + 40 [fdisk]
  8,16   0      273    21.924828821  4691  C  RA 10485704 + 40 [0]
  8,16   0      274    21.925034843  4691  Q  RA 4096 + 8 [fdisk]
  8,16   0      275    21.925035922  4691  G  RA 4096 + 8 [fdisk]
  8,16   0      276    21.925036136  4691  P   N [fdisk]
  8,16   0      277    21.925036332  4691  U   N [fdisk] 1
  8,16   0      278    21.925036608  4691  I  RA 4096 + 8 [fdisk]
  8,16   0      279    21.925037460  4691  D  RA 4096 + 8 [fdisk]
  8,16   0      280    21.925060546  4691  C  RA 4096 + 8 [0]
  8,16   0      281    21.926699960  4691  D   N 0 [fdisk]
  8,16   0      282    21.926734457  4691  C   N [0]
  8,16   0      283    21.926741052  4691  D   R 8 [fdisk]
  8,16   0      284    21.926810169  4691  C   R [0]
  8,16   0      285    21.926817998  4691  D   R 4 [fdisk]
  8,16   0      286    21.926874628  4691  C   R [0]
  8,16   0      287    21.926880654  4691  D   R 4 [fdisk]
  8,16   0      288    21.926924167  4691  C   R [65415]
  8,16   0      289    21.926967928  4691  Q   R 0 + 8 [fdisk]
  8,16   0      290    21.926969856  4691  G   R 0 + 8 [fdisk]
  8,16   0      291    21.926970617  4691  I   R 0 + 8 [fdisk]
  8,16   0      292    21.926996497   322  D   R 0 + 8 [kworker/0:1H]
  8,16   0      293    21.927044961   322  C   R 0 + 8 [0]
  8,16   1        1    21.927190948  4691  Q   R 8 + 8 [fdisk]
  8,16   1        2    21.927192513  4691  G   R 8 + 8 [fdisk]
  8,16   1        3    21.927192966  4691  I   R 8 + 8 [fdisk]
  8,16   1        4    21.927200092   320  D   R 8 + 8 [kworker/1:1H]
  8,16   0      294    21.927438278     0  C   R 8 + 8 [0]
  8,16   1        5    21.927621236  4691  Q   R 24 + 8 [fdisk]
  8,16   1        6    21.927622724  4691  G   R 24 + 8 [fdisk]
  8,16   1        7    21.927623249  4691  I   R 24 + 8 [fdisk]
  8,16   1        8    21.927628936   320  D   R 24 + 8 [kworker/1:1H]
  8,16   0      295    21.927892181     0  C   R 24 + 8 [0]
CPU0 (sdb):
 Reads Queued:          41,     1040KiB	 Writes Queued:           0,        0KiB
 Read Dispatches:       41,     1040KiB	 Write Dispatches:        0,        0KiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:       43,     1048KiB	 Writes Completed:        0,        0KiB
 Read Merges:            0,        0KiB	 Write Merges:            0,        0KiB
 Read depth:             1        	 Write depth:             0
 PC Reads Queued:        0,        0KiB	 PC Writes Queued:        0,        0KiB
 PC Read Disp.:          4,        0KiB	 PC Write Disp.:          0,        0KiB
 PC Reads Req.:          0		 PC Writes Req.:          0
 PC Reads Compl.:        4		 PC Writes Compl.:        0
 IO unplugs:            40        	 Timer unplugs:           0
CPU1 (sdb):
 Reads Queued:           2,        8KiB	 Writes Queued:           0,        0KiB
 Read Dispatches:        2,        8KiB	 Write Dispatches:        0,        0KiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:        0,        0KiB	 Writes Completed:        0,        0KiB
 Read Merges:            0,        0KiB	 Write Merges:            0,        0KiB
 Read depth:             1        	 Write depth:             0
 IO unplugs:             0        	 Timer unplugs:           0

Total (sdb):
 Reads Queued:          43,     1048KiB	 Writes Queued:           0,        0KiB
 Read Dispatches:       43,     1048KiB	 Write Dispatches:        0,        0KiB
 Reads Requeued:         0		 Writes Requeued:         0
 Reads Completed:       43,     1048KiB	 Writes Completed:        0,        0KiB
 Read Merges:            0,        0KiB	 Write Merges:            0,        0KiB
 PC Reads Queued:        0,        0KiB	 PC Writes Queued:        0,        0KiB
 PC Read Disp.:          4,        0KiB	 PC Write Disp.:          0,        0KiB
 PC Reads Req.:          0		 PC Writes Req.:          0
 PC Reads Compl.:        4		 PC Writes Compl.:        0
 IO unplugs:            40        	 Timer unplugs:           0

Throughput (R/W): 47KiB/s / 0KiB/s
Events (sdb): 303 entries
Skips: 0 forward (0 -   0.0%)

fio输出示例（）

root@ub1804:/home/xiaoyue# fio --name=reply --filename=/dev/sdb --direct=1 --read_iolog=sdb.bin
reply: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.1
Starting 1 process
Jobs: 1 (f=1): [R(1)][0.8%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 47m:40s]
reply: (groupid=0, jobs=1): err= 0: pid=4709: Thu May 30 17:38:31 2024
   read: IOPS=1, BW=47.8KiB/s (48.9kB/s)(1048KiB/21939msec)
    clat (usec): min=162, max=2913, avg=321.81, stdev=431.20
     lat (usec): min=162, max=2913, avg=322.20, stdev=431.22
    clat percentiles (usec):
     |  1.00th=[  163],  5.00th=[  172], 10.00th=[  174], 20.00th=[  182],
     | 30.00th=[  198], 40.00th=[  200], 50.00th=[  202], 60.00th=[  212],
     | 70.00th=[  235], 80.00th=[  297], 90.00th=[  578], 95.00th=[  717],
     | 99.00th=[ 2900], 99.50th=[ 2900], 99.90th=[ 2900], 99.95th=[ 2900],
     | 99.99th=[ 2900]
   bw (  KiB/s): min=   16, max=   16, per=34.04%, avg=16.00, stdev= 0.00, samples=1
   iops        : min=    4, max=    4, avg= 4.00, stdev= 0.00, samples=1
  lat (usec)   : 250=72.09%, 500=16.28%, 750=6.98%, 1000=2.33%
  lat (msec)   : 4=2.33%
  cpu          : usr=0.16%, sys=0.00%, ctx=90, majf=0, minf=66
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=43,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=47.8KiB/s (48.9kB/s), 47.8KiB/s-47.8KiB/s (48.9kB/s-48.9kB/s), io=1048KiB (1073kB), run=21939-21939msec

Disk stats (read/write):
  sdb: ios=2/0, merge=0/0, ticks=1/0, in_queue=0, util=0.04%
root@ub1804:/home/xiaoyue#