From
3. NVIDIA MLX5 Compress Driver — Data Plane Development Kit 23.11.0 documentation (dpdk.org)
5. NVIDIA MLX5 Common Driver
Note
NVIDIA acquired Mellanox Technologies in 2020. The DPDK documentation and code might still include instances of or references to Mellanox trademarks (like BlueField and ConnectX) that are now NVIDIA trademarks.
The mlx5 common driver library (librte_common_mlx5) provides support for NVIDIA ConnectX-4, NVIDIA ConnectX-4 Lx, NVIDIA ConnectX-5, NVIDIA ConnectX-6, NVIDIA ConnectX-6 Dx, NVIDIA ConnectX-6 Lx, NVIDIA ConnectX-7, NVIDIA BlueField, NVIDIA BlueField-2 and NVIDIA BlueField-3 families of 10/25/40/50/100/200 Gb/s adapters.
Information and documentation for these adapters can be found on the NVIDIA website. Help is also provided by the NVIDIA Networking forum. In addition, there is a web section dedicated to DPDK.
5.1. Design
For security reasons and to enhance robustness, this driver only handles virtual memory addresses. The way resources allocations are handled by the kernel, combined with hardware specifications that allow handling virtual memory addresses directly, ensure that DPDK applications cannot access random physical memory (or memory that does not belong to the current process).
There are different levels of objects and bypassing abilities which are used to get the best performance:
- Verbs is a complete high-level generic API
- Direct Verbs is a device-specific API
- DevX allows accessing firmware objects
- Direct Rules manages flow steering at the low-level hardware layer
On Linux, above interfaces are provided by linking with libibverbs and libmlx5. See Linux Prerequisites for installation.
On Windows, DevX is the only requirement from the above list. See Windows Prerequisites for DevX SDK package installation.
5.2. Classes
One mlx5 device can be probed by a number of different PMDs. To select a specific PMD, its name should be specified as a device parameter (e.g. 0000:08:00.1,class=eth
).
In order to allow probing by multiple PMDs, several classes may be listed separated by a colon. For example: class=crypto:regex
will probe both Crypto and RegEx PMDs.
5.2.1. Supported Classes
class=compress
for NVIDIA MLX5 Compress Driver.class=crypto
for NVIDIA MLX5 Crypto Driver.class=eth
for NVIDIA MLX5 Ethernet Driver.class=regex
for NVIDIA MLX5 RegEx Driver.class=vdpa
for NVIDIA MLX5 vDPA Driver.
By default, the mlx5 device will be probed by the eth
PMD.
5.2.2. Limitations
eth
andvdpa
PMDs cannot be probed at the same time. All other combinations are possible.- On Windows, only
eth
andcrypto
are supported.
5.3. Compilation Prerequisites
5.3.1. Linux Prerequisites
This driver relies on external libraries and kernel drivers for resources allocations and initialization. The following dependencies are not part of DPDK and must be installed separately:
-
libibverbs
User space Verbs framework used by
librte_common_mlx5
. This library provides a generic interface between the kernel and low-level user space drivers such aslibmlx5
.It allows slow and privileged operations (context initialization, hardware resources allocations) to be managed by the kernel and fast operations to never leave user space.
-
libmlx5
Low-level user space driver