交叉编译RF_RING

最新推荐文章于 2023-04-06 16:21:11 发布

转载最新推荐文章于 2023-04-06 16:21:11 发布 · 1.2k 阅读

2 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/mahj/p/8392500.html

文章标签：

#嵌入式 #移动开发

PF_RING介绍：

PF_RING是Luca研究出来的基于Linux内核级的高效数据包捕获技术。简单来说PF_RING 是一个高速数据包捕获库,通过它可以实现将通用 PC 计算机变成一个有效且便宜的网络测量工具箱,进行数据包和现网流量的分析和操作。同时支持调用用户级别的API来创建更有效的应用程序。现在我们知道PF_RING是拥有一套完整开发接口的高速数据包捕捉库，与我们熟知的libpcap十分相似，但其性能要优于libpcap

PF_RING下载（2018年1月）

https://www.ntop.org/get-started/download/#PF_RING

编译：

cd PF_RING

make

交叉编译：

背景：项目中需要在一个openwrt环境下植入PF_RING动态库。首先是生产此库。

平台：编译平台为ubuntu12.04, 交叉编译平台为高通ipq4019, arm7;

步骤：

1.建立package/utils/pfring，文件结构如下；

2.将网上下载RF_RING下的文件拷贝到src目录下，文件结构如下；

├── pfring
│   ├── Makefile
│   └── src
│       ├── ChangeLog
│       ├── doc
│       ├── drivers
│       ├── kernel
│       ├── LICENSE
│       ├── Makefile
│       ├── package
│       ├── README
│       ├── README.FIRST
│       ├── README.md
│       ├── tools
│       └── userland

3. pfring/Makefile如下：

include $(TOPDIR)/rules.mk

PKG_NAME:=pfring
PKG_RELEASE:=1
PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)

include $(INCLUDE_DIR)/package.mk

define Package/pfring
  SECTION:=utils
  CATEGORY:=Utilities
  TITLE:=pf_ring -- output libpfring.so
endef

define Package/pfring/description
    It's my first package demo.
endef

define Build/Prepare
        echo "Here is Package/Prepare"
        mkdir -p $(PKG_BUILD_DIR)
        $(CP) ./src/* $(PKG_BUILD_DIR)/
endef

define Package/pfring/install
        echo "Here is Package/install"
#       $(INSTALL_DIR) $(1)/bin
#       $(INSTALL_BIN) $(PKG_BUILD_DIR)/pfring $(1)/bin/
        $(INSTALL_DIR) $(1)/usr/lib
        $(CP) $(PKG_BUILD_DIR)/userland/deplib/* $(1)/usr/lib
        $(CP) $(PKG_BUILD_DIR)/userland/lib/*.so $(1)/usr/lib
endef

$(eval $(call BuildPackage,pfring))
~                                                                                                                                                                                                            
~

View Code

4. 修改src下makefile，使用交叉工具链

主Makefile如下，

all:
        cd kernel; make
        cd userland; make
        cd drivers; make

clean:
        cd kernel; make clean
        cd userland; make clean
        cd drivers; make clean
        -cd userland/snort/pfring-daq-module; make clean

snort:
        cd userland/snort/pfring-daq-module; autoreconf -ivf; ./configure; make
        cd userland/snort/pfring-daq-module-zc; autoreconf -ivf; ./configure; make

changelog:
        git log --since={`curl -s https://sourceforge.net/projects/ntop/files/PF_RING/|grep -o "<td headers=\"files_date_h\" class=\"opt\"><abbr title=\"[^\"]*\">[^<]*</abbr></td>"|head -n 3|tail -n 1|egrep -o "[0-9]+\-[0-9]+\-[0-9]+"|head -n 1`} --name-only --pretty=format:" - %s" > ./doc/Changelog.txt

documentation:
        cd doc; make
        cd doc/doxygen; doxygen Doxyfile

此处主要使用make all和make clean；使用交叉编译工具链，需要在make 后加入参数如下：

make CROSS_COMPILE=arm-openwrt-linux-uclibcgnueabi- ARCH=arm

整过过程是逢山开路，遇水搭桥，不过过程艰辛。有问题的可以私聊。主要遇到的问题归结如下

错误1

CC [M] /home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel/pf_ring.o

cc1: note: someone does not honour COPTS correctly, passed 0 times

In file included from include/asm-generic/int-ll64.h:10:0,

from ./arch/arm/include/asm/types.h:4,

from include/uapi/linux/types.h:4,

from include/linux/types.h:5,

from include/linux/list.h:4,

from include/linux/module.h:9,

from /home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel/pf_ring.c:78:

include/uapi/asm-generic/int-ll64.h:11:29: fatal error: asm/bitsperlong.h: No such file or directory

compilation terminated.

make[6]: *** [/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel/pf_ring.o] Error 1

make[5]: *** [_module_/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel] Error 2

make[5]: Leaving directory `/usr/src/linux-headers-4.4.0-31-generic'

make[4]: *** [all] Error 2

make[4]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel'

make[3]: *** [all] Error 2

make[3]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring'

make[2]: *** [/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/.built] Error 2

make[2]: Leaving directory `/home/box/work/xitong/qca/qca.git/qsdk/package/utils/pfring'

make[1]: *** [package/utils/pfring/compile] Error 2

make[1]: Leaving directory `/home/box/work/xitong/qca/qca.git/qsdk'

make: *** [package/utils/pfring/compile] Error 2

1）在vi kernel/pf_ring.c下增加kernel/pf_ring.c

2）在qca/src/linux-3.14/include/ uapi/asm-generic/int-ll64.h修改

//#include <asm/bitsperlong.h>

#include <asm-generic/bitsperlong.h>

思路：此处一直认为是内核库里没有bitsperlong.h文件，随着查资料才知道是因为没有使用交叉编译工具导致。如果使用平台的编译工具，编译时用的是gcc，如果使用交叉编译环境，编译工具为arm-openwrt-linux-uclibcgnueabi-gcc;

在PF_RING中需要修改交叉编译变量，方法如下,在make中加入参数CROSS_COMPILE=arm-openwrt-linux-uclibcgnueabi- ARCH=arm

Make à

Make CROSS_COMPILE=arm-openwrt-linux-uclibcgnueabi- ARCH=arm

错误2

include/uapi/linux/types.h:4:23: fatal error: asm/types.h: No such file or directory

#include <asm/types.h>

compilation terminated.

make[6]: *** [/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel/pf_ring.o] Error 1

make[5]: *** [_module_/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel] Error 2

make[5]: Leaving directory `/usr/src/linux-headers-3.11.0-15-generic'

make[4]: *** [all] Error 2

make[4]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel'

make[3]: *** [all] Error 2

make[3]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring'

make[2]: *** [/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/.built] Error 2

make[2]: Leaving directory `/home/pon/qca/qca.git/qsdk/package/utils/pfring'

make[1]: *** [package/utils/pfring/compile] Error 2

make[1]: Leaving directory `/home/pon/qca/qca.git/qsdk'

有以下解释

http://blog.youkuaiyun.com/luckywang1103/article/details/38112281

在linux中包含头文件时，有不少类似如下的写法：#include<asm/types.h>，但在linux kernel的include文件夹下并没有这个文件夹，倒是有不少以asm打头的文件夹，如：asm_powerpc、asm_ppc等。

源代码中没有asm这个文件夹，asm是一个符号连接，只有在你的主makefile的ARCH 变量赋值，编译内核的时候根据ARCH 的配置，

临时的生成asm文件夹及其下的文件，然后指向对应的体系结构的文件，如/include/asm-arm/ ，/include/asm-x86/

这样#include <asm/types.h> ＝＝/include/asm-x86/types.h

错误1，2的解决方法：

上面两个问题是编译kernel没有编译通过：

修改makefile中的make如下：

/home/pon/qca/qca.git/qsdk/package/utils/pfring/src/kernel/Makefile:

修改前：

$(MAKE) -C $(KERNEL_DIR) SUBDIRS=$(PWD) modules

修改后：

make CROSS_COMPILE=arm-openwrt-linux-uclibcgnueabi- ARCH=arm -C /home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/linux-ipq806x/linux-3.14.77/ M=/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/kernel EXTRA_CFLAGS='${EXTRA_CFLAGS}' modules

注意：此处存在隐患，如果工程更改了位置，此处需要路径需要修改；

错误3

checking for gcc... arm-openwrt-linux-uclibcgnueabi-gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... configure: error: in `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/userland':

configure: error: cannot run C compiled programs.

If you meant to cross compile, use `--host'.

See `config.log' for more details

make[4]: *** [lib/Makefile] Error 1

make[4]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/userland'

make[3]: *** [all] Error 2

make[3]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring'

make[2]: *** [/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/.built] Error 2

make[2]: Leaving directory `/home/pon/qca/qca.git/qsdk/package/utils/pfring'

make[1]: *** [package/utils/pfring/compile] Error 2

make[1]: Leaving directory `/home/pon/qca/qca.git/qsdk'

make: *** [package/utils/pfring/compile] 错误 2

原因：

没有编译平台信息；

解决：

在编译平台上运行uname -m，查看平台信息，将其放在--host=后面

修改文件package/utils/pfring/src/userland/Makefile，增加--host=arm

lib/Makefile:

./configure --host=arm

错误4

ar rs libpfring.a pfring.o pfring_mod.o pfring_utils.o pfring_mod_stack.o pfring_hw_filtering.o pfring_hw_timestamp.o pfring_mod_sysdig.o `ar t ../nbpf/libnbpf.a | grep -F .o | tr '\n' ' '`

ranlib libpfring.a

=*= making library libpfring.so =*=

gcc -Wl,-soname,libpfring.so.1 -g -shared pfring.o pfring_mod.o pfring_utils.o pfring_mod_stack.o pfring_hw_filtering.o pfring_hw_timestamp.o pfring_mod_sysdig.o `ar t ../nbpf/libnbpf.a | grep -F .o | tr '\n' ' '` -lpthread -lrt -ldl -lnl-genl-3 -lnl-3 -o libpfring.so

/usr/bin/ld: error: cannot find -lnl-genl-3

/usr/bin/ld: error: cannot find -lnl-3

collect2: ld returned 1 exit status

make[5]: *** [libpfring.so] Error 1

make[5]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/userland/lib'

make[4]: *** [libpfring] Error 2

make[4]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/userland'

make[3]: *** [all] Error 2

make[3]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring'

make[2]: *** [/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/.built] Error 2

make[2]: Leaving directory `/home/pon/qca/qca.git/qsdk/package/utils/pfring'

make[1]: *** [package/utils/pfring/compile] Error 2

make[1]: Leaving directory `/home/pon/qca/qca.git/qsdk'

make: *** [package/utils/pfring/compile] 错误 2

思路：

提示缺少两个库libnl-genl-3.so，libnl-3.so

pon@ubuntu:~/qca/qca.git/qsdk/build_dir$ find -name libnl-genl-3.so

./target-arm_cortex-a7_uClibc-1.0.14_eabi/libnl-3.2.21/ipkg-install/usr/lib/libnl-genl-3.so

./target-arm_cortex-a7_uClibc-1.0.14_eabi/libnl-3.2.21/lib/.libs/libnl-genl-3.so

pon@ubuntu:~/qca/qca.git/qsdk/build_dir$

pon@ubuntu:~/qca/qca.git/qsdk/build_dir$ find -name libnl-3.so

./target-arm_cortex-a7_uClibc-1.0.14_eabi/libnl-3.2.21/ipkg-install/usr/lib/libnl-3.so

./target-arm_cortex-a7_uClibc-1.0.14_eabi/libnl-3.2.21/lib/.libs/libnl-3.so

pon@ubuntu:~/qca/qca.git/qsdk/build_dir$

搜索文档发现是有有这两个库，只是编译过程中没有找到这两个库；

在package/utils/pfring/src/userland/configure：3605行通过-L加入.so绝对路径，编程错误提示

unsupported ELF machine number 40

考虑此处应该引用编译平台的库文件。在系统/usr/lib下没有找到libnl-genl-3.so和libnl-3.so，通过下面命令按照这两个库

sudo apt-get install libnl-3-dev

sudo apt-get install libnl-genl-3-dev

然后编译通过，编译后运行

file pfring/userland/lib/libpfring.so

pfring/userland/lib/libpfring.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=0x889bffb8f6b7d868873ea99fa9cfd5a0cd91a8e0, not stripped

说明没有通过交叉编译生产lib库；

在package/utils/pfring/src/userland/Makefile中make增加参数

make CROSS_COMPILE=arm-openwrt-linux-uclibcgnueabi- ARCH=arm

错误5

编译后

cc1: note: someone does not honour COPTS correctly, passed 0 times

=*= making library libpfring.a =*=

ar rs libpfring.a pfring.o pfring_mod.o pfring_utils.o pfring_mod_stack.o pfring_hw_filtering.o pfring_hw_timestamp.o pfring_mod_sysdig.o `ar t ../nbpf/libnbpf.a | grep -F .o | tr '\n' ' '`

ranlib libpfring.a

=*= making library libpfring.so =*=

arm-openwrt-linux-uclibcgnueabi-gcc -Wl,-soname,libpfring.so.1 -g -shared pfring.o pfring_mod.o pfring_utils.o pfring_mod_stack.o pfring_hw_filtering.o pfring_hw_timestamp.o pfring_mod_sysdig.o `ar t ../nbpf/libnbpf.a | grep -F .o | tr '\n' ' '` -lpthread -lrt -ldl -lnl-genl-3 -lnl-3 -o libpfring.so

/home/pon/qca/qca.git/qsdk/staging_dir/toolchain-arm_cortex-a7_gcc-4.8-linaro_uClibc-1.0.14_eabi/lib/gcc/arm-openwrt-linux-uclibcgnueabi/4.8.3/../../../../arm-openwrt-linux-uclibcgnueabi/bin/ld: nbpf_mod_rdif.o: Relocations in generic ELF (EM: 3)

nbpf_mod_rdif.o: error adding symbols: File in wrong format

collect2: error: ld returned 1 exit status

make[5]: *** [libpfring.so] Error 1

make[5]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/userland/lib'

make[4]: *** [libpfring] Error 2

make[4]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/userland'

make[3]: *** [all] Error 2

make[3]: Leaving directory `/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring'

make[2]: *** [/home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/.built] Error 2

make[2]: Leaving directory `/home/pon/qca/qca.git/qsdk/package/utils/pfring'

make[1]: *** [package/utils/pfring/compile] Error 2

make[1]: Leaving directory `/home/pon/qca/qca.git/qsdk'

make: *** [package/utils/pfring/compile] 错误 2

原因：

file pfring/userland/lib/nbpf_mod_rdif.o，

nbpf_mod_rdif.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not strippe

说明没有按照交叉编译编译生成nbpf_mod_rdif.o

解决：

修改/pfring/src/userland/nbpf/Makefile文件“

CC=arm-openwrt-linux-uclibcgnueabi-gcc

错误6

编译成功后，将其拷贝到/usr/lib下会出现下面错误：

install -d -m0755 /home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/ipkg-ipq806x/pfring/usr/lib

cp -fpR /home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/userland/lib/*.so /home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/ipkg-ipq806x/pfring/usr/lib

find /home/...../build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/pfring/ipkg-ipq806x/pfring -name 'CVS' -o -name '.svn' -o -name '.#*' -o -name '*~'| xargs -r rm -rf

Package pfring is missing dependencies for the following libraries:

libnl-3.so.200

libnl-genl-3.so.200

libpthread.so.1

librt.so.1

解决方法：

在/usr/lib下有libnl-3.so.200，libnl-genl-3.so.200，libpthread.so.1，librt.so.1，但需要建立软连接；

这四个文件存在于文件系统中，如下：

build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/root-ipq806x/usr/lib/libnl-3.so.200

uild_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/root-ipq806x/usr/lib/libnl-genl-3.so.200

build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/root-ipq806x/lib/libpthread.so.1

build_dir/target-arm_cortex-a7_uClibc-1.0.14_eabi/root-ipq806x/lib/librt.so.1