How do I configure a network interface bandwidth limitation?

最新推荐文章于 2021-05-06 20:48:52 发布

转载最新推荐文章于 2021-05-06 20:48:52 发布 · 452 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：https://access.redhat.com/solutions/40824

文章标签：

#NIC #bandwidth #limitation

本文介绍如何在Red Hat Enterprise Linux中配置网络接口的带宽限制，包括检查现有队列纪律、计算速率限制参数、应用速率限制到接口、确认配置及测试确认等步骤。适用于需要控制网络流量的系统或服务器。

https://access.redhat.com/solutions/40824

SOLUTION 已验证 - 已更新 2016年十二月22日14:58 -

English

环境

Red Hat Enterprise Linux
Networking

问题

How do I configure a network interface bandwidth limitation?
How to ratelimit a NIC?
How can we stop a system or server sending so much LAN traffic?

决议

Inspect Existing qdisc

Ensure the existing queueing discipline on the network interface is pfifo_fast or noqueue:

Raw

# ip link | grep noqueue
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT 
2: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT

Raw

# tc -s qdisc
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 3485442 bytes 40223 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0

If there is already a classless or classful qdisc applied, this needs to be removed with tc qdisc del, refer to man tc for full syntax of the various qdiscs.

Calculate Ratelimit Parameters

Units are described further on the man tc section PARAMETERS, common values for reference:

Raw

kbit       Kilobits per second
mbit       Megabits per second
gbit       Gigabits per second

kb or k    Kilobytes
mbit       Megabits
mb or m    Megabytes

The rate can be expressed as: rate 20mbit (20 megabit)

The buffer needs to be at least 1 kilobyte per megabit, this buffer is oversized: buffer 256kb (256 kilobytes)

Select how long you wish a packet to sit in the buffer before being dropped: latency 100ms (100 milliseconds)

The exact values for your network will need to be determined through configuration and testing. It is recommended to test with both bandwidth testing tools such as iperf and with the actual production workload to ensure desired outcomes are met.

Apply the Ratelimit to the Interface

We use the Token Bucket Filter to apply the ratelimit:

Raw

# tc qdisc add dev eth0 root tbf rate 20mbit buffer 256kb latency 100ms

Confirm Configuration

We can see qdisc tbf is now applied:

Raw

# tc -s qdisc
qdisc tbf 8001: dev eth0 root refcnt 2 rate 20000Kbit burst 256Kb lat 100.0ms 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0

Test and Confirm

A bandwidth test shows we're reaching roughly our desired bandwidth:

Raw

# iperf -l 1M -w 4M -f m -t 20 -c 172.16.1.7
------------------------------------------------------------
Client connecting to 172.16.1.7, TCP port 5001
TCP window size: 8.00 MByte (WARNING: requested 4.00 MByte)
------------------------------------------------------------
[  3] local 172.16.1.6 port 61900 connected with 172.16.1.7 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-20.8 sec  53.0 MBytes  21.4 Mbits/sec

The qdisc has been working to limit traffic it submits to the NIC:

Raw

# tc -s qdisc
qdisc tbf 8001: dev eth0 root refcnt 2 rate 20000Kbit burst 256Kb lat 100.0ms 
 Sent 2344194 bytes 1561 pkt (dropped 19, overlimits 130675 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0

To Persist Across Reboots

The methods described at How to make NIC ethtool settings persistent (apply automatically at boot) can also be used to persist these changes.

In RHEL 5, RHEL 6, and RHEL 7 without NetworkManager, write /sbin/ifup-local to apply the ratelimit at interface start.

In RHEL 7 with NetworkManager, write a NM dispatcher script as described in man NetworkManager.

Removal

If you wish to remove the TBF limitation, the qdisc can be deleted with:

Raw

# tc qdisc del dev eth0 root

The interface will revert back to its default queueing discipline:

Raw

# tc -s qdisc
qdisc pfifo_fast 0: dev eth0 root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 0bit 0pps backlog 0b 0p requeues 0

Troubleshooting

Ratelimiting will never be 100% accurate, but within 10% accuracy is a reasonable expectation. Faster ratelimits (eg: gigabit) will likely be more accurate than lower ratelimits (eg: kilobit).
Note that the TBF can only restrict what we send out. It is possible to ratelimit incoming traffic with moderate accuracy, as described at How to limit the bandwidth of incoming packets.
If the ratelimit encountered in testing is far off, NIC offloading features may need to be disabled, as described at tc command's transport rate does not seem normal.