ONNX模型解析推理工具onnx-parser使用记录

本文介绍了一个国内开发者创建的ONNX模型解析及推理工具onnx-parser,该工具已集成到RT-Thread开发包中。文章详细记录了onnx-parser的安装、编译过程,并分析了其使用方法和原理。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

onnx-parser是国内开发者开发的一个ONNX网络模型解析以及推理工具,当前已经被纳入了RT-Thread的开发包,由于RT-Thread的ENV环境支持,下面分析记录一下它的使用和原理分析。

首先下载代码:

git clone https://github.com/wuhanstudio/onnx-parser

下载Scons工具

Scons是一个开放源码、以Python语言编码的自动化构建工具。可以将SCons看作是经典Make实用程序的改进的、跨平台的替代品.

修改编译错误:

代码下载下来,默认是编译不过的,会出现下面的错误:

解决方式是修改其中的 onnx.pb-c.h实现:

diff --git a/onnx.pb-c.h b/onnx.pb-c.h
index 0fe2a2e..22a76f2 100644
--- a/onnx.pb-c.h
+++ b/onnx.pb-c.h
@@ -4,7 +4,7 @@
 #ifndef PROTOBUF_C_src_2fonnx_2eproto__INCLUDED
 #define PROTOBUF_C_src_2fonnx_2eproto__INCLUDED
 
-#include <protobuf-c.h>
+#include <protobuf-c/protobuf-c.h>
 
 PROTOBUF_C__BEGIN_DECLS
 

编译:

进入到example目录下:

​caozilong@caozilong-Vostro-3268:~/Workspace/onnx/onnx-parser/example$ scons 
scons: Reading SConscript files ...

scons: warning: Two different environments were specified for target /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.o,
	but they appear to have the same action: $CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
File "/home/caozilong/Workspace/onnx/onnx-parser/example/SConstruct", line 18, in <module>

scons: warning: Two different environments were specified for target /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.o,
	but they appear to have the same action: $CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
File "/home/caozilong/Workspace/onnx/onnx-parser/example/SConstruct", line 18, in <module>

scons: warning: Two different environments were specified for target /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.o,
	but they appear to have the same action: $CC -o $TARGET -c $CFLAGS $CCFLAGS $_CCCOMCOM $SOURCES
File "/home/caozilong/Workspace/onnx/onnx-parser/example/SConstruct", line 18, in <module>
scons: done reading SConscript files.
scons: Building targets ...
gcc -o backend/add.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/add.c
gcc -o backend/conv2d.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/conv2d.c
gcc -o backend/dense.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/dense.c
gcc -o backend/info.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/info.c
gcc -o backend/matmul.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/matmul.c
gcc -o backend/maxpool.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/maxpool.c
gcc -o backend/model.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/model.c
gcc -o backend/relu.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/relu.c
gcc -o backend/softmax.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/softmax.c
gcc -o backend/transpose.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend backend/transpose.c
gcc -o mnist/mnist.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend mnist/mnist.c
gcc -o mnist/mnist_model.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend mnist/mnist_model.c
gcc -o mnist/mnist_sm.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend mnist/mnist_sm.c
gcc -o /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.c
gcc -o /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.c
gcc -o /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.c
gcc -o onnx-mnist /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.o /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.o /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.o mnist/mnist.o backend/add.o backend/conv2d.o backend/dense.o backend/info.o backend/matmul.o backend/maxpool.o backend/model.o backend/relu.o backend/softmax.o backend/transpose.o -lm
gcc -o onnx-mnist-model /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.o /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.o /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.o mnist/mnist_model.o backend/add.o backend/conv2d.o backend/dense.o backend/info.o backend/matmul.o backend/maxpool.o backend/model.o backend/relu.o backend/softmax.o backend/transpose.o -lm
gcc -o onnx-mnist-sm /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.o /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.o /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.o mnist/mnist_sm.o backend/add.o backend/conv2d.o backend/dense.o backend/info.o backend/matmul.o backend/maxpool.o backend/model.o backend/relu.o backend/softmax.o backend/transpose.o -lm
gcc -o parse/parse_test.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend parse/parse_test.c
gcc -o onnx-parser /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.o /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.o /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.o parse/parse_test.o
gcc -o transpose/transpose_test.o -c -I. -I/home/caozilong/Workspace/onnx/onnx-parser -Ibackend transpose/transpose_test.c
gcc -o onnx-transpose /home/caozilong/Workspace/onnx/onnx-parser/onnx-parser.o /home/caozilong/Workspace/onnx/onnx-parser/onnx.pb-c.o /home/caozilong/Workspace/onnx/onnx-parser/protobuf-c.o transpose/transpose_test.o backend/transpose.o backend/info.o -lm
scons: done building targets.
caozilong@caozilong-Vostro-3268:~/Workspace/onnx/onnx-parser/example$ 

验证:

./onnx-parser mnist-sm.onnx

./onnx-mnist

./onnx-mnist-sm 

 ./onnx-mnist-model

backend目录中包含了算子的实现

另外一个算子库:

git clone https://github.com/wuhanstudio/onnx-backend

caozilong@caozilong-Vostro-3268:~/Workspace/onnx/onnx-backend$ tree
.
├── examples
│   ├── mnist.c
│   ├── mnist.h
│   ├── mnist_model.c
│   ├── mnist_sm.c
│   └── model
│       ├── mnist-keras.ipynb
│       ├── mnist-lg.onnx
│       └── mnist-sm.onnx
├── README.md
├── SConscript
└── src
    ├── add.c
    ├── conv2d.c
    ├── dense.c
    ├── info.c
    ├── matmul.c
    ├── maxpool.c
    ├── model.c
    ├── onnx.h
    ├── relu.c
    ├── softmax.c
    └── transpose.c

3 directories, 20 files
caozilong@caozilong-Vostro-3268:~/Workspace/onnx/onnx-backend$

网络模型结构:


程序打印出来的网络结构: 


规律,输入图像都进行了归一化

结束!

### 使用 TensorRT 进行 ONNX 模型推理 #### 1. TensorRT 的简介 TensorRT 是 NVIDIA 提供的一个高性能深度学习推理库,能够优化神经网络模型并将其部署到生产环境。它支持多种框架导出的模型文件格式,其中包括 ONNX 格式。 为了使用 TensorRT 对 ONNX 模型进行推理,通常需要完成以下几个主要步骤: --- #### 2. 安装依赖项 在 Ubuntu 18.04 上安装 CUDA 和 PyCUDA 是必要的前提条件之一[^1]。以下是具体操作流程: - **安装 TensorRT**: 可通过官方文档获取适合版本的 TensorRT 软件包,并按照说明完成安装。 - **安装 pycuda**: `pip install pycuda` 命令即可轻松实现 Python 环境下的 GPU 编程接口绑定。 --- #### 3. 将 PyTorch 模型转换为 ONNX 格式 如果原始模型是以 PyTorch 构建,则需先将其保存为 ONNX 文件形式以便后续处理。下面是一个简单的代码片段展示这一过程: ```python import torch import torchvision dummy_input = torch.randn(1, 3, 224, 224).cuda() model = torchvision.models.resnet50(pretrained=True).cuda() torch.onnx.export(model, dummy_input, "resnet50.onnx", verbose=False) print("Model successfully converted to ONNX format.") ``` 上述脚本会生成名为 resnet50.onnx 的新文件用于下一步解析与编译工作流之中[^2]. --- #### 4. 利用 TensorRT 解析并序列化 ONNX 模型 一旦拥有了有效的 .onnx 文件之后,就可以借助 TensorRT API 来创建对应的引擎实例对象了。这里给出一段典型示范程序如下所示: ```python from tensorrt import Logger, Runtime, Builder, NetworkDefinitionCreationFlag, \ OnnxParserFlags, IBuilderConfig, PrecisionMode import numpy as np def build_engine(onnx_file_path="resnet50.onnx"): TRT_LOGGER = Logger(Logger.WARNING) with Builder(TRT_LOGGER) as builder,\ builder.create_network(NetworkDefinitionCreationFlag.EXPLICIT_BATCH) as network,\ Runtime(TRT_LOGGER) as runtime: config = builder.create_builder_config() profile = builder.create_optimization_profile() parser = trt.OnnxParser(network, TRT_LOGGER) if not os.path.exists(onnx_file_path): raise FileNotFoundError(f"The provided ONNX file does not exist at {onnx_file_path}") with open(onnx_file_path, 'rb') as model: if not parser.parse(model.read()): for error in range(parser.num_errors): print(parser.get_error(error)) return None input_shape = (1, 3, 224, 224) min_shapes = {"input": tuple([1]+list(input_shape)[1:])} opt_shapes = {"input": tuple([4]+list(input_shape)[1:])} max_shapes = {"input": tuple([8]+list(input_shape)[1:])} profile.set_shape('input', list(min_shapes.values())[0], list(opt_shapes.values())[0], list(max_shapes.values())[0]) config.add_optimization_profile(profile) config.max_workspace_size = 1 << 30 # 设置最大 workspace 大小为 1GB config.set_flag(IBuilderConfig.FLAG_FP16) serialized_engine = builder.build_serialized_network(network, config) engine = runtime.deserialize_cuda_engine(serialized_engine) return engine ``` 该函数定义了一个构建器方法来加载指定路径上的 ONNX 文件并通过设置不同的 batch size 参数范围来自动生成最佳性能配置方案^. --- #### 5. 执行推理任务 最后一步就是实际调用之前建立好的 inference 引擎来进行前向传播计算得出最终结果啦! ```python class HostDeviceMem(object): """Simple helper data class that's a little nicer to use than a 2-tuple.""" def __init__(self, host_mem, device_mem): self.host = host_mem self.device = device_mem def __str__(self): return "Host:\n" + str(self.host) + "\nDevice:\n" + str(self.device) def allocate_buffers(engine): inputs = [] outputs = [] bindings = [] stream = cuda.Stream() for binding in engine: size = trt.volume(engine.get_binding_shape(binding)) * engine.max_batch_size dtype = trt.nptype(engine.get_binding_dtype(binding)) # Allocate host and device buffers. host_mem = cuda.pagelocked_empty(size, dtype) device_mem = cuda.mem_alloc(host_mem.nbytes) # Append the device buffer to device bindings. bindings.append(int(device_mem)) # Append to appropriate lists based on whether it is an input or output. if engine.binding_is_input(binding): inputs.append(HostDeviceMem(host_mem, device_mem)) else: outputs.append(HostDeviceMem(host_mem, device_mem)) return inputs, outputs, bindings, stream def do_inference(context, bindings, inputs, outputs, stream, batch_size=1): [cuda.memcpy_htod_async(inp.device, inp.host, stream) for inp in inputs] context.execute_async(batch_size=batch_size, bindings=bindings, stream_handle=stream.handle) [cuda.memcpy_dtoh_async(out.host, out.device, stream) for out in outputs] stream.synchronize() engine = build_engine() context = engine.create_execution_context() inputs, outputs, bindings, stream = allocate_buffers(engine) image = cv2.imread('test.jpg') image_resized = preprocess_image(image) # 自己编写预处理逻辑 np.copyto(inputs[0].host, image_resized.ravel()) do_inference(context=context, bindings=bindings, inputs=inputs, outputs=outputs, stream=stream, batch_size=1) output_data = outputs[0].host.reshape(-1) result = postprocess(output_data) # 同样自己实现后处理部分 ``` 注意这里的 preprocess_image 函数以及 postprocess 方法都需要依据具体的业务需求自行开发完善才行哦[^4]! --- #### 6. 数据类型的注意事项 当从图像或其他源读取数据时,请务必确认其数值精度满足目标模型的要求。例如,默认情况下 TorchScript 导出后的 ONNX 结构期望接收 float32 类别的张量作为输入;然而 OpenCV 或 PIL 库所返回的结果往往是 uint8 形式的数组结构体。因此,在送入任何样本给定之前应当执行显式的类型强制转换动作像这样子做:`image.astype(np.float32)`[^3]. ---
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

papaofdoudou

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值