paddleServing目标检测深度学习模型在线部署教程

一、准备环境

1.ubuntu16.04
2.python3.6
3.nvidia-docker2安装，参考链接(https://blog.youkuaiyun.com/weixin_40699155/article/details/106441047)
4.$ pip install paddle-serving-client -i https://mirror.baidu.com/pypi/simple

二、模型转换

模型必须是由save_inference_model接口存储。

1.利用PaddleDetection开发套件中export_serving_model.py

 $ python tools/export_serving_model.py \
          -c configs/yolov4/yolov4_cspdarknet_voc.yml \
          --output_dir=./inference_model \
          -o weights=output/yolov4_cspdarknet_voc/best_model

2.直接转化

  import paddle.fluid as fluid
  import paddle_serving_client.io as serving_io
  
  place = fluid.CUDAPlace(0)
  exe = fluid.Executor(place)
  #获取模型文件
  save_path="/path/to/model/"

  [infer_program,feeded_var_names,target_var]=fluid.io.load_inference_model(dirname=save_path,executor=exe)
  print("input",feeded_var_names)
  print("output",target_var)
  
  def build_input():
   image = fluid.layers.data(name='image', shape=image_shape, dtype='float32')
   im_info = fluid.layers.data(name="im_info", shape=[3], dtype='float32')
   im_shape = fluid.layers.data(name="im_shape", shape=[3], dtype='float32')
   return [image,im_shape]
  feed_list = build_input()
  feed_dict = {"image":feed_list[0],"im_info":feed_list[1]"im_shape":feed_list[2]}
  fetch_dict ={"outputs":target_var[0]}
  
  #保存serving需要的模型与配置文件，参数1是server端模型与配置文件保存的目录，参数2是client端配置文件保存的目录，参数3是输入dict，参数4是输出dict，参数5是模型的program
  serving_io.save_model("serving_server", "serving_client",
         feed_dict, fetch_dict, infer_program)

三、在线部署

1.拉取镜像

$ nvidia-docker pull hub.baidubce.com/paddlepaddle/serving:latest-gpu

2.启动镜像

$ nvidia-docker run -p 9292:9292 --name test -td hub.baidubce.com/paddlepaddle/serving:latest-gpu

3.进入镜像

$ docker exec -it test bash

4.安装paddle-serving-server-gpu

$ pip3 install paddle-serving-server-gpu -i https://mirror.baidu.com/pypi/simple

5.启动服务

注意：首次启动服务时，会自动安装paddle_serving_server_gpu模块

RPC service:
    $ python3 -m paddle_serving_server_gpu.serve --model serving_server --thread 10 --port 9292 --mem_optim true --ir_optim true >std.log 2>err.log &
参数说明：
    --model 指定部署模型文件夹路径
    --thread 使用线程
    --port 对外开放端口，这个要与启动镜像时的端口相同
    --mem_optim 开启内存/显存优化
    --ir_optim 开启计算图分析优化，包括OP融合等
    & 后端执行

四、远程测试

1) 安装预处理模块paddle_serving_app

   $ pip install paddle_serving_app -i https://mirror.baidu.com/pypi/simple

2) 测试代码

  from paddle_serving_client import Client
  from paddle_serving_app.reader import *
  import numpy as np
  
  图像预处理初始化
  preprocess = Sequential([
   File2Image(), BGR2RGB(), Div(255.0),
   Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225], False),
   Resize(800, 1333,1), Transpose((2, 0, 1)), PadStride(32)
  ])  
  图像后期处理初始化
  postprocess = RCNNPostprocess("label_list.txt", "output")  
  创建socket
  client = Client()
  client.load_client_config("serving_client/serving_client_conf.prototxt")
  连接socket
  client.connect(['127.0.0.1:10002'])
  im = preprocess('111.jpg')
  发出请求
  fetch_map = client.predict(feed={"image": im, "im_info": np.array(list(im.shape[1:]) + [1.0]),
           "im_shape": np.array(list(im.shape[1:]) + [1.0])}, fetch=["multiclass_nms_0.tmp_0"])
  fetch_map["image"] = '111.jpg'
  图像后期处理，测试结果最终保存在当前文件夹的output文件夹下
  postprocess(fetch_map)

注意：paddle_serving_app提供的图像预处理或后期处理结果可能与自己训练的模型存在偏差，需要到python安装包path/to/python3.6/site-packages/paddle_serving_app/reader/下修改image_reader.py文件