编译DPDK20与21以及testpmd和pktgen的测试

悠扬侠

已于 2023-11-10 10:17:17 修改

阅读量2.4k

点赞数 1

分类专栏： DPDK 文章标签： linux 服务器 performance 网络 c语言

于 2022-12-09 22:45:00 首次发布

本文链接：https://blog.youkuaiyun.com/weixin_39950873/article/details/128240016

版权

DPDK 专栏收录该内容

8 篇文章

订阅专栏

书接上文，本文描述DPDK20与21跑testpmd和pktgen测试的步骤。和DPDK19不同的是，DPDK20与21没有dpdk-setup.sh这个脚本工具了，感觉不是很方便欸。

1. Setup

1.1 Build DPDK

下载和解压DPDK之后，进入DPDK文件夹。

1.1.1 meson build

#meson build

1.1.2 ninja -C build

#ninja -C build

运行之后的最后一行是这样，运行的时候就是0到2479的过程。

[2479/2479] Linking target app/dpdk-testpmd.

1.1.3 ninja install

#cd build
#ninja install

运行之后最后一部分如下。

'./librte_raw_ioat.so' -> 'dpdk/pmds-21.0/librte_raw_ioat.so'
'./librte_raw_ioat.so.21' -> 'dpdk/pmds-21.0/librte_raw_ioat.so.21'
'./librte_raw_ioat.so.21.0' -> 'dpdk/pmds-21.0/librte_raw_ioat.so.21.0'
'./librte_raw_ntb.so' -> 'dpdk/pmds-21.0/librte_raw_ntb.so'
'./librte_raw_ntb.so.21' -> 'dpdk/pmds-21.0/librte_raw_ntb.so.21'
'./librte_raw_ntb.so.21.0' -> 'dpdk/pmds-21.0/librte_raw_ntb.so.21.0'
'./librte_raw_octeontx2_dma.so' -> 'dpdk/pmds-21.0/librte_raw_octeontx2_dma.so'
'./librte_raw_octeontx2_dma.so.21' -> 'dpdk/pmds-21.0/librte_raw_octeontx2_dma.so.21'
'./librte_raw_octeontx2_dma.so.21.0' -> 'dpdk/pmds-21.0/librte_raw_octeontx2_dma.so.21.0'
'./librte_raw_octeontx2_ep.so' -> 'dpdk/pmds-21.0/librte_raw_octeontx2_ep.so'
'./librte_raw_octeontx2_ep.so.21' -> 'dpdk/pmds-21.0/librte_raw_octeontx2_ep.so.21'
'./librte_raw_octeontx2_ep.so.21.0' -> 'dpdk/pmds-21.0/librte_raw_octeontx2_ep.so.21.0'
'./librte_raw_skeleton.so' -> 'dpdk/pmds-21.0/librte_raw_skeleton.so'
'./librte_raw_skeleton.so.21' -> 'dpdk/pmds-21.0/librte_raw_skeleton.so.21'
'./librte_raw_skeleton.so.21.0' -> 'dpdk/pmds-21.0/librte_raw_skeleton.so.21.0'
'./librte_regex_octeontx2.so' -> 'dpdk/pmds-21.0/librte_regex_octeontx2.so'
'./librte_regex_octeontx2.so.21' -> 'dpdk/pmds-21.0/librte_regex_octeontx2.so.21'
'./librte_regex_octeontx2.so.21.0' -> 'dpdk/pmds-21.0/librte_regex_octeontx2.so.21.0'
'./librte_vdpa_ifc.so' -> 'dpdk/pmds-21.0/librte_vdpa_ifc.so'
'./librte_vdpa_ifc.so.21' -> 'dpdk/pmds-21.0/librte_vdpa_ifc.so.21'
'./librte_vdpa_ifc.so.21.0' -> 'dpdk/pmds-21.0/librte_vdpa_ifc.so.21.0'

1.1.4 ldconfig

#ldconfig

注意，这里需要权限，可以root也可以sudo。

1.2 Bind PMD

vfio和uio选择一个即可。

1.2.1 使用vfio

#sudo modprobe vfio-pci

/* 看看IOMMU是否enable */
#cat /sys/module/vfio/parameters/enable_unsafe_noiommu_mode 
N
/* 如果没有就需要enable */
#sudo echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode

/* down网卡*/
#ifconfig enp1s0 down

查看网卡的状态：

#./usertools/dpdk-devbind.py --status

Network devices using kernel driver
===================================
0000:01:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' if=enp1s0 drv=r8169 unused=vfio-pci 
0000:21:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' if=enp33s0 drv=r8169 unused=vfio-pci 
0000:64:00.1 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' if=enp100s0f1 drv=r8169 unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

Crypto devices using kernel driver
==================================
0000:23:00.1 'Starship/Matisse Cryptographic Coprocessor PSPCPP 1486' drv=ccp unused=vfio-pci 

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

绑定网卡。

/* dpdk dev bind */
#./usertools/dpdk-devbind.py --bind=vfio-pci 01:00.0 21:00.0

# ./usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:01:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' drv=vfio-pci unused=r8169
0000:21:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' drv=vfio-pci unused=r8169

Network devices using kernel driver
===================================
0000:64:00.1 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' if=enp100s0f1 drv=r8169 unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

Crypto devices using kernel driver
==================================
0000:23:00.1 'Starship/Matisse Cryptographic Coprocessor PSPCPP 1486' drv=ccp unused=vfio-pci 

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

1.2.2 使用uio

一来DPDK更推荐vfio，另一方面使用igb_uio更加复杂了，大家可以自行百度。

1.3 Create hugepages

#sudo mkdir -p /dev/hugepages
#sudo mountpoint -q /dev/hugepages || mount -t hugetlbfs nodev /dev/hugepages
#sudo echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages;

2. Run testpmd

这部分比较简单了，和之前的差不太多。

#./build/app/dpdk-testpmd -- -i

EAL: Detected 48 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 8 (No-IOMMU)
EAL: Ignore mapping IO port bar(0)
EAL: Probe PCI driver: net_r8168 (10ec:8168) device: 0000:01:00.0 (socket 0)
r8168: driver version 1.001 Dec  8 2022 17:18:20 init
EAL: Ignore mapping IO port bar(0)
EAL: Probe PCI driver: net_r8168 (10ec:8168) device: 0000:21:00.0 (socket 0)
r8168: driver version 1.001 Dec  8 2022 17:18:20 init
EAL: No legacy callbacks, legacy socket not created
Interactive-mode selected
testpmd: create a new mbuf pool <mb_pool_0>: n=523456, size=2176, socket=0
testpmd: preferred mempool ops selected: ring_mp_mc
Configuring Port 0 (socket 0)
Port 0: 00:E0:4C:68:00:9A
Configuring Port 1 (socket 0)
Port 1: 00:E0:4C:68:00:06
Checking link statuses...
Done
testpmd> start tx_first 
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP allocation mode: native
Logical Core 1 (socket 0) forwards packets on 2 streams:
  RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
  RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00

  io packet forwarding packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=2
  port 0: RX queue number: 1 Tx queue number: 1
    Rx offloads=0x0 Tx offloads=0x0
    RX queue: 0
      RX desc=512 - RX free threshold=64
      RX threshold registers: pthresh=0 hthresh=0  wthresh=0
      RX Offloads=0x0
    TX queue: 0
      TX desc=512 - TX free threshold=0
      TX threshold registers: pthresh=0 hthresh=0  wthresh=0
      TX offloads=0x0 - TX RS bit threshold=0
  port 1: RX queue number: 1 Tx queue number: 1
    Rx offloads=0x0 Tx offloads=0x0
    RX queue: 0
      RX desc=512 - RX free threshold=64
      RX threshold registers: pthresh=0 hthresh=0  wthresh=0
      RX Offloads=0x0
    TX queue: 0
      TX desc=512 - TX free threshold=0
      TX threshold registers: pthresh=0 hthresh=0  wthresh=0
      TX offloads=0x0 - TX RS bit threshold=0
testpmd> stop
Telling cores to stop...
Waiting for lcores to finish...

  ---------------------- Forward statistics for port 0  ----------------------
  RX-packets: 2139190        RX-dropped: 0             RX-total: 2139190
  TX-packets: 2139188        TX-dropped: 0             TX-total: 2139188
  ----------------------------------------------------------------------------

  ---------------------- Forward statistics for port 1  ----------------------
  RX-packets: 2139200        RX-dropped: 0             RX-total: 2139200
  TX-packets: 2139203        TX-dropped: 0             TX-total: 2139203
  ----------------------------------------------------------------------------

  +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
  RX-packets: 4278390        RX-dropped: 0             RX-total: 4278390
  TX-packets: 4278391        TX-dropped: 0             TX-total: 4278391
  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.

3. Run pktgen

下载pktgen的时候需要注意pktgen的版本需要与DPDK的版本匹配，比如DPDK21需要pktgen 22.2.0，否则会报错。

/* 进入pktgen的目录下 */
#cd pktgen-21.03.1/
 
/* 设置DPDK源码包的绝对路径。当编译外部应用时，用户必须定义这个环境变量。 */
#export RTE_SDK=/home/Driver/dpdk-stable-20.11.6

/* compile pktgen */ 
#meson build
#ninja -C build

/* 这里加了个-T，不然看起来乱乱的 */
# ./build/app/pktgen -l 0-3 -n 3 -- -P -T -m"1.0,2.1"