RKNN部署记录-1

导出模型：

将tensorflow模型导出为.pb文件

tf.keras.models.save_model(model,"./models/pb_/")

将.pb文件转换为.onnx文件（命令行操作）

python -m tf2onnx.convert --saved-model ./pb_ --opset 18 --output model.onnx

python下查看onnx结构： 

import onnx

def print_graph_nodes(model_path):
    # 加载 ONNX 模型
    model = onnx.load(model_path)

    # 遍历所有图节点并打印节点信息
    for node in model.graph.node:
        node_type = node.op_type
        node_name = node.name
        print(f'Node Type: {node_type}, Node Name: {node_name}')

if __name__ == '__main__':
    onnx_model_file = './models/model.onnx'
    print_graph_nodes(onnx_model_file)

测试onnx模型：

import onnxruntime
# import numpy as np
# input1 = np.random.random((1, 7, 7, 3)).astype('float32')

ort_sess = onnxruntime.InferenceSession("./models/model.onnx")
ort_inputs = {ort_sess.get_inputs()[0].name: input1 }
ort_outs = ort_sess.run(None, ort_inputs)

RKNN环境搭建：ubuntu 22.04.4，Python3.10，RKNN-Toolkit2-2.0.0，

在~/.config/pip/pip.conf文件中设置镜像以下载tensorflow-2.8.0：

index-url = https://pypi.tuna.tsinghua.edu.cn/simple

依次执行

sudo apt-get install python3 python3-dev python3-pip 

sudo apt-get install libxslt1-dev zlib1g zlib1g-dev libglib2.0-0 libsm6 libgl1-mesa-glx libprotobuf-dev gcc

pip3 install -r doc/requirements_cp310.txt

ip3 install packages/rknn_toolkit2-2.0.0b0+9bab5682-cp310-cp310-linux_x86_64.whl

参考：RKNN例程，RKNN工具

 使用rknn库转换模型（Linux系统）

实际部署后，将动态输入修改为静态，由于python API没有动态接口的讲解和示例，实际运行也不成功。

同时去掉量化：

rknn.config(dynamic_input = [[[1,600,1]]],
            target_platform = 'rk3568')

ret = rknn.load_onnx(model = './model.onnx')

ret = rknn.build(do_quantization=True, dataset='./dataset.txt')

改为：

rknn.config(target_platform = 'rk3568')

ret = rknn.load_onnx(model = './model.onnx', inputs = ['input'], input_size_list = [[1,600,1]])

ret = rknn.build(do_quantization=False)

 

 原代码：

import numpy as np

from rknn.api import RKNN
rknn = RKNN(verbose=True)



rknn.config(dynamic_input = [[[1,600,1]]],
            target_platform = 'rk3568')

if __name__ == '__main__':

    # Load model
    print('--> Loading model')

			
    #ret = rknn.load_tflite(model = './quantized_saved_model.tflite')
    ret = rknn.load_onnx(model = './model.onnx')
    if ret != 0:
        print('Load model failed!')
        exit(ret)
    print('done')


    # Build model
    print('--> Building model')
    ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
    if ret != 0:
        print('Build model failed!')
        exit(ret)
    print('done')

    # Export rknn model
    print('--> Export rknn model')
    ret = rknn.export_rknn('./mobilenet_v1.rknn')
    if ret != 0:
        print('Export rknn model failed!')
        exit(ret)
    print('done')
    
    
    
rknn.release()

 待解决问题：模型中使用了talking head attention，模型使用tf.sinsum构建。tf.sinsum会导致构建RKNN模型失败，报错提示使用了动态图。

p_attn_talk = tf.convert_to_tensor(tf.einsum('ij,ajbk->aibk', self._post_softmax_weight, p_attn))

 解决方案：

        修改为：

_pre_softmax_weight = np.random.randint(0, 10, size=(8, 8))
scores = np.random.randint(0, 10, size=(1, 8, 300, 300))

scores_rhp = tf.reshape(scores,(tf.shape(scores)[0],tf.shape(scores)[1],tf.shape(scores)[2]*tf.shape(scores)[3]))
print(scores_rhp.shape)
scores_talk = tf.matmul(_pre_softmax_weight, scores_rhp)
print(scores_talk.shape)
scores_talk = tf.reshape(scores_talk,(tf.shape(scores)[0],tf.shape(scores)[1],tf.shape(scores)[2],tf.shape(scores)[3]))
print(scores_talk.shape)

scores_talk_1 = tf.convert_to_tensor(tf.einsum('ij,ajbk->aibk', _pre_softmax_weight, scores))

print("\n结果是否相等：", np.allclose(scores_talk.numpy(), scores_talk_1.numpy()))

修改后的模型可以继续使用原始权重数据，效果基本不变。