scatter/gather IO原理

本文探讨了在Linux环境下使用readv和writev系统调用的优点,特别是对于非连续内存块的数据操作。通过实例说明了如何利用这些调用进行原子性的I/O操作,避免了数据复制的开销,并确保了多个数据段的连续写入不会被其他进程的写入操作所干扰。

  https://stackoverflow.com/questions/10520182/linux-when-to-use-scatter-gather-io-readv-writev-vs-a-large-buffer-with-frea

 

The main convenience offered by readvwritev is:

  1. It allows working with non contiguous blocks of data. i.e. buffers need not be part of an array, but separately allocated.
  2. The I/O is 'atomic'. i.e. If you do a writev, all the elements in the vector will be written in one contiguous operation, and writes done by other processes will not occur in between them.

e.g. say, your data is naturally segmented, and comes from different sources:

struct foo *my_foo;
struct bar *my_bar;
struct baz *my_baz;

my_foo = get_my_foo();
my_bar = get_my_bar();
my_baz = get_my_baz();

Now, all three 'buffers' are not one big contiguous block. But you want to write them contiguously into a file, for whatever reason (say for example, they are fields in a file header for a file format).

If you use write you have to choose between:

  1. Copying them over into one block of memory using, say, memcpy (overhead), followed by a single write call. Then the write will be atomic.
  2. Making three separate calls to write (overhead). Also, write calls from other processes can intersperse between these writes (not atomic).

If you use writev instead, its all good:

  1. You make exactly one system call, and no memcpy to make a single buffer from the three.
  2. Also, the three buffers are written atomically, as one block write. i.e. if other processes also write, then these writes will not come in between the writes of the three vectors.

So you would do something like:

struct iovec iov[3];

iov[0].iov_base = my_foo;
iov[0].iov_len = sizeof (struct foo);
iov[1].iov_base = my_bar;
iov[1].iov_len = sizeof (struct bar);
iov[2].iov_base = my_baz;
iov[2].iov_len = sizeof (struct baz);

bytes_written = writev (fd, iov, 3);

Sources:

  1. http://pubs.opengroup.org/onlinepubs/009604499/functions/writev.html
  2. http://linux.die.net/man/2/readv

https://www.oreilly.com/library/view/linux-system-programming/9781449341527/ch04.html

评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值