YOLO入门教程（二）——OpenVINO™部署YOLO目标检测模型与异步推理实现150FPS+实时检测【含教程源码 + 环境配置】

引言

YOLO（You Only Look Once）作为一个目标检测算法，其调用的接口为Python语言，通常情况下需要将YOLO模型的pt格式转换为其他格式的模型供其他编程语言调用，常见的onnx格式可以支持C++/C#/Python等多种语言的调用，本文教程主要介绍如何将YOLO模型的pt格式转换为onnx格式，并在Window环境下用C#调用模型并推理。

OpenVINO框架为.NET 6.0需要安装Visual Studio2022

前期准备

VIP可以直接下载项目源码开始前建议先下载参考源码，在Anaconda中重新配置一个Python3.10的环境，安装的PackageList如下：

CUDA版本相同的情况下这里建议直接参考作者所配置的环境版本

根据上面的表格可以查到安装CUDA11.8版本的是正确的

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

要先安装pytorch再安装yolo环境

Step1 pt格式转onnx格式

from ultralytics import YOLO
import os

# Model Path
save_model_name = os.path.normpath(os.path.join(os.getcwd(), './yolov10-main/runs/detect/mytrain_0806/weights/best.pt'))

# Load model
model = YOLO(os.path.normpath(save_model_name))

# Create 'best.onnx'
model.export(format="onnx", dynamic=False, simplify=True)

save_model_name替换成pt文件所在的路径
生成的onnx文件也在save_model_name路径下

Step2 创建工程文件并安装NuGet Package

Step3 预处理API部署YOLO模型

将OpenVINO部署方式封装为接口，只需要实例化Predictor类即可加载模型

using OpenCvSharp.Dnn;
using OpenCvSharp;
using OpenVinoSharp;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using OpenVinoSharp.preprocess;
using System.Diagnostics;
using OpenVinoSharp.Extensions;
using OpenVinoSharp.Extensions.model;

namespace DeepLearningDotNet
{
   
    public class Predictor : IDisposable
    {
   
        /// <summary>
        /// OpenVINO Runtime Core
        /// </summary>
        private Core core;
        /// <summary>
        /// 加载前处理API的模型
        /// </summary>
        private Model model;
        /// <summary>
        /// 加载到设备中的模型
        /// </summary>
        private CompiledModel compiled;
        /// <summary>
        /// OpenVINO推理接口
        /// </summary>
        public InferRequest openvino_api_infer;
        /// <summary>
        /// OpenCV推理接口
        /// </summary>
        public Net opencv_infer;
        /// <summary>
        /// 接口类型
        /// </summary>
        private string engine;
        /// <summary>
        /// 模型图片尺寸
        /// </summary>
        private int[] input_size;

        public Predictor() {
    }

        /// <summary>
        /// 实例化推理器
        /// </summary>
        /// <param name="model_path"></param>
        /// <param name="engine"></param>
        /// <param name="device"></param>
        /// <exception cref="ArgumentNullException"></exception>
        public Predictor(string model_path, string engine = "OpenVINO", string device = "AUTO", int[] input_size = null)
        {
   
            //判断路径是否合法
            if (model_path == null || model_path == "" || !File.Exists(model_path))
            {
   
                throw new ArgumentNullException(nameof(model_path));
            }

            this.engine = engine;
            if (engine == "OpenVINO")
            {
   
                // -------- Step 1. Initialize OpenVINO Runtime Core --------
                core = new Core();
                //判断设备是否可用
                if (!core.get_available_devices().Contains(device))
                {
   
                    throw new ArgumentNullException(nameof(device));
                }
                // -------- Step 2. Read inference model --------
                Model tempModel = core.read_model(model_path);
                OvExtensions.printf_model_info(tempModel);
                PrePostProcessor processor = new PrePostProcessor(tempModel);
                this.input_size = input_size ?? new int[] {
    1, 3, 640, 640 };
                Tensor input_tensor_pro = new Tensor(new OvType(ElementType.U8), new Shape(this.input_size[0], this.input_size[2], this.input_size[3], this.input_size[1]));
                InputInfo input_info = processor.input(0);
                InputTensorInfo input_tensor_info = input_info.tensor();
                input_tensor_info.set_from(input_tensor_pro).set_layout(new Layout("NHWC")).set_color_format(ColorFormat.BGR);
                PreProcessSteps process_steps = input_info.preprocess();
                process_steps.convert_color(ColorFormat.RGB).resize(ResizeAlgorithm.RESIZE_LINEAR)
                   .convert_element_type(new OvType(ElementType.F32)).scale(255.0f).convert_layout(new Layout("NCHW"));
                model = processor.build();
                // -------- Step 3. Loading a model to the device --------
                compiled = core.compile_model(model, device);
                // -------- Step 4. Create an infer request --------
                openvino_api_infer = compiled.create_infer_request();

            }
            if (engine == "OpenCv")
            {
   
                opencv_infer = CvDnn.ReadNetFromOnnx(model_path);
            }
        }

        public void Dispose()
        {
   
            openvino_api_infer.Dispose();
            opencv_infer.Dispose();
            compiled.Dispose();
            model.Dispose();
            core.Dispose();
            GC.Collect();
        }

        /// <summary>
        /// OpenVINO推理方法
        /// </summary>
        /// <param name="input_data"></param>
        /// <param name="input_names"></param>
        /// <param name="input_size"></param>
        /// <param name="output_names"></param>
        /// <param name="output_sizes"></param>
        /// <returns></returns>
        public List<float[]> OpenVinoInfer(Mat img, List<string> input_names, int[] input_size, List<string> output_names, List<int