opencv C++部署crnn车牌文字识别模型

前言

首先该文章是基于大佬的文章的车牌文字识别部分，模型的训练以及python代码参考大佬github，原文链接：车牌识别算法

由于大佬原文只有python的检测代码，而我需要C++的代码，所以参考大佬的python代码做了opencv C++的代码，流程 比较简单：1.需要做图片的前处理，具体就是将CWH转为NCHW。2.进行推理。3.进行后处理，后处理比较简单，找到推理数据中最大的数据就行。

首先需要说的是我的模型输入是1x3x48x168，输出是1x21x78

具体如下：

1.图片前处理

cv::Mat rec_pre_processing(cv::Mat img, cv::Size size = cv::Size(168, 48)){
    float mean_value = 0.588; 
    float std_value = 0.193;

    cv::resize(img, img, size);
    
    img.convertTo(img, CV_32F); // covert to float32
    img = img / 255 - mean_value;
    img = img / std_value;
    printf("%d--%d--%d\n",img.rows,img.cols,img.channels());
    std::vector<cv::Mat> inMatList;
    inMatList.push_back(img);
    cv::Mat ret = cv::dnn::blobFromImages(inMatList,1.0,size,cv::Scalar(0, 0, 0),true, false, CV_32F);
    cv::Mat img_reshape;
    
    return ret;

}

2.推理及后处理

/// @brief 初始化字符vec主要用作后续显示
/// 这样写其实比较傻，不过懒得改了
std::vector<std::string> initialize_string(void){
    std::vector<std::string> vec_tmp;
    std::string str[] = {"#","京","沪","津","渝","冀","晋","蒙","辽","吉","黑","苏","浙","皖","闽","赣",
                        "鲁","豫","鄂","湘","粤","桂","琼","川","贵","云","藏","陕","甘","青","宁","新",
                        "学","警","港","澳","挂","使","领","民","航","危","0","1","2","3","4","5","6","7",
                        "8","9","A","B","C","D","E","F","G","H","J","K","L","M","N","P","Q","R","S","T",
                        "U","V","W","X","Y","Z","险","品"};
    int num = sizeof(str)/sizeof(str[0]);
    std::vector<std::string> vec_tmp;
    for (int n = 0; n < num; n++) {
        vec_tmp.push_back(str[n]);
    }
    return vec_tmp;
}

/// @brief 解码函数
std::string decodePlate(std::vector<int> preds) {
    int pre = 0;
    std::vector<int> newPreds;
    for (int i = 0; i < preds.size(); i++) {
        if (preds[i] != 0 && preds[i] != pre) {
            newPreds.push_back(preds[i]);
        }
        pre = preds[i];
    }
    std::string plate = "";
    std::vector<std::string> vec_string = initialize_string();
    for (int i = 0; i < newPreds.size(); i++) {
        plate += vec_string[newPreds[i]];
        // printf("%s\n",vec_string[newPreds[i]]);

    }
    return plate;
}

/// @brief 推理并获取结果
std::string get_plate_result(cv::Mat img, cv::dnn::Net& session_rec) {
    /* 对输入图片进行预处理 */
    cv::Mat img_processed = rec_pre_processing(img);

   /* 运行 ONNX 模型 */
    std::vector<cv::Mat> outputs;
    session_rec.setInput(img_processed);
    session_rec.forward(outputs);

    /* 获取输出 */
    cv::Mat y_onnx = outputs[0];

    std::vector<std::vector<float>> vec_float;
    vec_float.resize(21);
   
    for (int i = 0; i < y_onnx.size[0]; ++i) {
        for (int j = 0; j < y_onnx.size[1]; ++j) {
            for (int k = 0; k < y_onnx.size[2]; ++k) {
                vec_float[j].push_back(y_onnx.at<float>(i, j, k));
            }
        }
    }

    std::vector<int> plate_no;
    plate_no.resize(21);
    plate_no.clear();
    for(int i = 0;i < 21;i++){
        auto num = getMaxIndex(vec_float[i]);
        plate_no.push_back(num);
    }
    
    std::string plate_num = decodePlate(plate_no);
    return plate_num;
}

int main(int argc,char** argv){
    cv::dnn::Net net = cv::dnn::readNet("best_crnn.onnx");
    cv::Mat in = cv::imread("tmp8724.png"/* ,0 */);
    std::string plate_string = get_plate_result(in,net);
    printf("plate:%s\n",plate_string.c_str());
    
}

具体测试可以截取一张车牌图片进行测试，不过需要提醒的是，这里只是文字识别，没有图像增强，所以图像质量差（偏暗、模糊等），倾斜度太大的图片是识别不出来的