在RHEL7上使用 Intel NVMe+LVM+XFS时大块读的性能问题

最新推荐文章于 2025-08-23 15:01:53 发布

转载最新推荐文章于 2025-08-23 15:01:53 发布 · 1.2k 阅读

文章标签：

#intel #nvme #xfs #lvm #large block read

RHEL7 同时被 2 个专栏收录

432 篇文章

订阅专栏

Linux

292 篇文章

订阅专栏

RHEL7中XFS文件系统在LVM和Intel NVMe设备上的大块读取性能不佳，此问题在直接使用NVMe设备的XFS或EXT4上未出现。通过RHSA-2018:3083更新解决，优化了stripwidth设置。

RHEL7: Large block read performance issue with XFS+LVM with Intel NVMe

https://access.redhat.com/solutions/3406851

SOLUTION 已验证 - 已更新 2019年二月28日23:06 -

English

环境

Red Hat Enterprise Linux 7
XFS on a LVM Logical Volume
NVMe storage

问题

Large block reads from a XFS file system backed by a LVM Logical Volume on a Intel NVMe device are under performing.
This behavior is not observed on a XFS file system directly on the NVMe device.
Reads from a EXT4 file system on a Logical Volume do not exhibit this behavior.

决议

Red Hat Enterprise Linux 7

This issue has been resolved with the errata RHSA-2018:3083 for the package(s) kernel-3.10.0-957.el7 or later.

根源

File systems were not being created with optimal strip width at time of creation. This is because # mkfs.xfs was not receiving accurate information from opt_io hint.
chunk_sectors are now properly reported to io_min and io_opt.
This now resolves the performance degradation.

诊断步骤

Results from testing

Raw

XFS+LVM:

   READ: bw=319MiB/s (335MB/s), 319MiB/s-319MiB/s (335MB/s-335MB/s), io=37.4GiB (40.2GB), run=120001-120001msec
   READ: bw=321MiB/s (336MB/s), 321MiB/s-321MiB/s (336MB/s-336MB/s), io=37.6GiB (40.3GB), run=120001-120001msec

XFS+LVM+MQ

# cat /sys/module/dm_mod/parameters/use_blk_mq 
Y

   READ: bw=321MiB/s (337MB/s), 321MiB/s-321MiB/s (337MB/s-337MB/s), io=37.6GiB (40.4GB), run=120001-120001msec


XFS:  Without LVM

   READ: bw=1178MiB/s (1235MB/s), 1178MiB/s-1178MiB/s (1235MB/s-1235MB/s), io=138GiB (148GB), run=120001-120001msec
   READ: bw=1067MiB/s (1119MB/s), 1067MiB/s-1067MiB/s (1119MB/s-1119MB/s), io=125GiB (134GB), run=120001-120001msec

EXT4: Without LVM

   READ: bw=1156MiB/s (1212MB/s), 1156MiB/s-1156MiB/s (1212MB/s-1212MB/s), io=135GiB (145GB), run=120001-120001msec

EXT4+LVM:

   READ: bw=1026MiB/s (1076MB/s), 1026MiB/s-1026MiB/s (1076MB/s-1076MB/s), io=120GiB (129GB), run=120002-120002msec