2. PP-ORCv3

2.1. PP-OCRv3简介

PP-OCR 是百度公布并开源的OCR领域算法，一个轻量级的OCR系统，在实现前沿算法的基础上，考虑精度与速度的平衡， 进行模型瘦身和深度优化，使其尽可能满足产业落地需求。

PP-OCR是一个两阶段的OCR系统，其中文本检测算法选用DB，文本识别算法选用CRNN，并在检测和识别模块之间添加文本方向分类器，以应对不同方向的文本识别。

PP-OCRv2在PP-OCR的基础上，进一步在5个方面重点优化，检测模型采用CML协同互学习知识蒸馏策略和CopyPaste数据增广策略； 识别模型采用LCNet轻量级骨干网络、UDML 改进知识蒸馏策略和Enhanced CTC loss损失函数改进（如上图红框所示）， 进一步在推理速度和预测效果上取得明显提升。

PP-OCRv3 在PP-OCRv2的基础上进一步升级。 整体的框架图保持了与PP-OCRv2相同的pipeline，针对检测模型和识别模型进行了优化。 其中，检测模块仍基于DB算法优化，而识别模块不再采用CRNN，换成了IJCAI 2022最新收录的文本识别算法SVTR，并对其进行产业适配。

最新的 PP-OCRv4 在PP-OCRv3的基础上进一步升级。 整体的框架图保持了与PP-OCRv3相同的pipeline，针对检测模型和识别模型进行了数据、网络结构、训练策略等多个模块的优化。

PaddleOCR 支持通过FastDeploy在RKNPU2上部署相关模型， 具体的模型列表参考下 这里 ， 接下来我们将在板卡上，使用FastDeploy部署测试PP-ORCv3。

提示

教程测试环境：lubancat-0/1/2使用Debian10，lubancat-4使用Debian11，PC端是WSL2（ubuntu20.04）

2.2. 环境安装

FastDeploy相关环境安装可以参考下前面 FastDeploy章节 或者参考下 FastDeploy文档 。

需要在PC或者云服务器等等安装PaddlePaddle、FastDeploy和paddle2onnx等工具，进行模型的训练和转换等操作；在板卡上安装FastDeploy运行环境（Pyrhon或者C++），进行模型的部署推理等操作。

2.3. 模型转换

PP-OCR系列模型训练参考 PaddleOCR教程文档 。

教程测试直接从PaddleOCR获取 模型，然后转换成RKNN模型。

在RKNPU2上部署PP-OCR系列模型时，我们需要把Paddle的推理模型转为RKNN模型， 但rknn_toolkit2工具暂不支持直接从Paddle直接转换为RKNN模型，因此需要先将Paddle推理模型转为ONNX模型, 最后转为RKNN模型。

在PC ubuntu20.04系统上（或者WSL2）：

# 下载PP-OCRv3文字检测模型，并解压
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_infer.tar
tar -xvf ch_PP-OCRv3_det_infer.tar

# 下载文字方向分类器模型，并解压
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
tar -xvf ch_ppocr_mobile_v2.0_cls_infer.tar

# 下载PP-OCRv3文字识别模型，并解压
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_infer.tar
tar -xvf ch_PP-OCRv3_rec_infer.tar

# 使用paddle2onnx, 将模型转换成ONNX模型
paddle2onnx --model_dir ch_PP-OCRv3_det_infer \
            --model_filename inference.pdmodel \
            --params_filename inference.pdiparams \
            --save_file ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det_infer.onnx \
            --enable_dev_version True

paddle2onnx --model_dir ch_ppocr_mobile_v2.0_cls_infer \
            --model_filename inference.pdmodel \
            --params_filename inference.pdiparams \
            --save_file ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.onnx \
            --enable_dev_version True

paddle2onnx --model_dir ch_PP-OCRv3_rec_infer \
            --model_filename inference.pdmodel \
            --params_filename inference.pdiparams \
            --save_file ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer.onnx \
            --enable_dev_version True

# 固定模型的输入shape
python -m paddle2onnx.optimize --input_model ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det_infer.onnx \
                            --output_model ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det_infer.onnx \
                            --input_shape_dict "{'x':[1,3,960,960]}"

python -m paddle2onnx.optimize --input_model ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.onnx \
                            --output_model ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.onnx \
                            --input_shape_dict "{'x':[1,3,48,192]}"

python -m paddle2onnx.optimize --input_model ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer.onnx \
                            --output_model ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer.onnx \
                            --input_shape_dict "{'x':[1,3,48,320]}"

获取工具源码：

# 获取FastDeploy源码，然后切换到examples/vision/ocr/PP-OCR/rockchip目录（也可以从配套例程获取）
git clone https://github.com/PaddlePaddle/FastDeploy.git
cd FastDeploy/examples/vision/ocr/PP-OCR/rockchip

可以修改下rknpu2_tools/config目录下的配置文件（一般默认），修改配置文件中的模型路径 model_path 和模型输出路径 output_folder ， 需要修改到对应的模型目录，三个模型配置文件都需要修改，项目以ppocrv3_cls.yaml为例：

rknpu2_tools/config/ppocrv3_cls.yaml

mean:
-
    - 127.5
    - 127.5
    - 127.5
std:
-
    - 127.5
    - 127.5
    - 127.5
model_path: ./ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.onnx
outputs_nodes:
do_quantization: False
dataset:
output_folder: "./ch_ppocr_mobile_v2.0_cls_infer"

然后使用export.py将前面转换的ONNX模型转成RKNN模型：

# 切换到rknpu2_tools的同级目录下，此时的目录结构：
.
├── ch_PP-OCRv3_det_infer
├── ch_PP-OCRv3_rec_infer
├── ch_ppocr_mobile_v2.0_cls_infer
└── rknpu2_tools

# 使用FastDeploy提供的工具，将ONNX模型转成RKNN模型（环境中需要安装rknn_toolkit2工具）
# --target_platform指定平台，lubancat-4是rk3588，lubancat-0/1/2是rk3566或者rk3568,,教程测试的是lubancat-4板卡
python rknpu2_tools/export.py --config_path rknpu2_tools/config/ppocrv3_det.yaml \
                            --target_platform rk3588

python rknpu2_tools/export.py --config_path rknpu2_tools/config/ppocrv3_rec.yaml \
                            --target_platform rk3588

python rknpu2_tools/export.py --config_path rknpu2_tools/config/ppocrv3_cls.yaml \
                            --target_platform rk3588

加载onnx模型过程可能会出现：

E load_onnx: onnx.onnx_cpp2py_export.checker.ValidationError: Field 'shape' of type is required but missing.

模型一些类型缺少“shape”，可以使用onnxruntime的工具重新推理下onnx的shape：

python symbolic_shape_infer.py --input ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer_old.onnx  \
                      --output ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.onnx

具体symbolic_shape_infer.py程序请查看 onnxruntime 。

最后使用export.py转换的RKNN模型保存在对应目录中。

2.4. Python部署测试

部署推理程序可以从 Fastdeploy 或者PaddleOCR源码中获取（或者参考教程配套例程）， 然后复制前面转换模型的文件夹到板卡，然后进行推理测试（测试平台是lubancat-4，Fastdeploy是develop分支）：

# 下载测试图片（也可以使用其他测试图片）和字典文件
wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/doc/imgs/12.jpg
wget https://gitee.com/paddlepaddle/PaddleOCR/raw/release/2.6/ppocr/utils/ppocr_keys_v1.txt

# 下载部署示例代码,从配套例程中获取就是rockchip/python目录下
git clone https://github.com/PaddlePaddle/FastDeploy.git
cd  FastDeploy/examples/vision/ocr/PP-OCR/rockchip/python

# 此时的目录文件
cat@lubancat:~/pp-ocrv3$ ls
12.jpg  ch_PP-OCRv3_det_infer  ch_PP-OCRv3_rec_infer  ch_ppocr_mobile_v2.0_cls_infer  cpp  ppocr_keys_v1.txt  python  visualized_result.jpg

执行cpu推理，使用onnx模型：

# CPU推理，
cat@lubancat:~/pp-ocrv3$ python3 python/infer.py \
                --det_model ./ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det_infer.onnx \
                --cls_model ./ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.onnx \
                --rec_model ./ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer.onnx \
                --rec_label_file ./ppocr_keys_v1.txt \
                --image 12.jpg \
                --device cpu
[INFO] fastdeploy/runtime/runtime.cc(326)::CreateOrtBackend     Runtime initialized with Backend::ORT in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(326)::CreateOrtBackend     Runtime initialized with Backend::ORT in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(326)::CreateOrtBackend     Runtime initialized with Backend::ORT in Device::CPU.
det boxes: [[274,174],[285,173],[285,178],[274,179]]rec text:  rec score:0.000000 cls label: 1 cls score: 0.832612
det boxes: [[43,408],[483,390],[483,431],[44,449]]rec text: 上海斯格威铂尔曼大酒店 rec score:0.890337 cls label: 0 cls score: 1.000000
det boxes: [[186,456],[399,448],[399,480],[186,488]]rec text: 打浦路15号 rec score:0.988907 cls label: 0 cls score: 1.000000
det boxes: [[18,501],[513,485],[514,537],[18,554]]rec text: 绿洲仕格维花园公寓 rec score:0.992894 cls label: 0 cls score: 1.000000
det boxes: [[78,553],[404,541],[404,573],[78,585]]rec text: 打浦路252935号 rec score:0.983241 cls label: 0 cls score: 1.000000

Visualized result save in ./visualized_result.jpg

执行NPU推理，使用rknn模型：

# NPU推理，测试平台是lubancat-4，都是指定rk3588平台的模型文件，如果是lubancat-0/1/2使用rk3566/rk3568平台的模型
cat@lubancat:~/pp-ocrv3$ python3 python/infer.py \
                --det_model ./ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det_infer_rk3588_unquantized.rknn \
                --cls_model ./ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v20_cls_infer_rk3588_unquantized.rknn \
                --rec_model ./ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer_rk3588_unquantized.rknn \
                --rec_label_file ppocr_keys_v1.txt \
                --image 12.jpg \
                --device npu
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.1b19 (32afb0e92@2023-07-14T12:46:17)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.8
index=0, name=x, n_dims=4, dims=[1, 960, 960, 3], n_elems=2764800, size=5529600, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=sigmoid_0.tmp_0, n_dims=4, dims=[1, 1, 960, 960], n_elems=921600, size=1843200, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend  Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.1b19 (32afb0e92@2023-07-14T12:46:17)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.8
index=0, name=x, n_dims=4, dims=[1, 48, 192, 3], n_elems=27648, size=55296, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_0.tmp_0, n_dims=2, dims=[1, 2, 0, 0], n_elems=2, size=4, fmt=UNDEFINED, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend  Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.1b19 (32afb0e92@2023-07-14T12:46:17)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.8
index=0, name=x, n_dims=4, dims=[1, 48, 320, 3], n_elems=46080, size=92160, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_5.tmp_0, n_dims=4, dims=[1, 40, 6625, 1], n_elems=265000, size=530000, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend  Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory
       The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory
       The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory
       The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
det boxes: [[276,174],[285,173],[285,178],[276,179]]rec text:  rec score:0.000000 cls label: 1 cls score: 0.766602
det boxes: [[43,408],[483,390],[483,431],[44,449]]rec text: 上海斯格威铂尔曼大酒店 rec score:0.888450 cls label: 0 cls score: 1.000000
det boxes: [[186,456],[399,448],[399,480],[186,488]]rec text: 打浦路15号 rec score:0.988769 cls label: 0 cls score: 1.000000
det boxes: [[18,501],[513,485],[514,537],[18,554]]rec text: 绿洲仕格维花园公寓 rec score:0.992676 cls label: 0 cls score: 1.000000
det boxes: [[78,553],[404,541],[404,573],[78,585]]rec text: 打浦路252935号 rec score:0.983398 cls label: 0 cls score: 1.000000

Visualized result save in ./visualized_result.jpg

推理结果图片显示（截取主要部分）：

2.5. C++部署测试

与前面python部署测试目录文件相同，然后进入cpp目录，编译程序：

# 部署推理程序同样可以从Fastdeploy或者PaddleOCR源码中获取（也可以从配套例程获取）

# 复制模型文件夹和一些测试图片，此时的目录文件
cat@lubancat:~/pp-ocrv3$ ls
11.jpg 12.jpg  ch_PP-OCRv3_det_infer  ch_PP-OCRv3_rec_infer  ch_ppocr_mobile_v2.0_cls_infer  cpp  ppocr_keys_v1.txt  python  visualized_result.jpg
cat@lubancat:~/pp-ocrv3$ cd cpp
cat@lubancat:~/pp-ocrv3/cpp$
cat@lubancat:~/pp-ocrv3/cpp$ mkdir build && cd build
cat@lubancat:~/pp-ocrv3/cpp/build$

# 执行cmake命令，FASTDEPLOY_INSTALL_DIR指定前面编译安装Fastdeploy C++ SDK的目录，教程测试就是~/FastDeploy/build/fastdeploy-0.0.0
cat@lubancat:~/pp-ocrv3/cpp/build$ cmake .. -DFASTDEPLOY_INSTALL_DIR=~/FastDeploy/build/fastdeploy-0.0.0
# 执行make命令编译
cat@lubancat:~/pp-ocrv3/cpp/build$ make -j
Scanning dependencies of target infer_demo
[ 50%] Building CXX object CMakeFiles/infer_demo.dir/infer.cc.o
[100%] Linking CXX executable infer_demo
[100%] Built target infer_demo

会在build目录下生成infer_demo推理程序，将该程序拷贝到模型文件同级目录下，然后测试：

# 目录文件
cat@lubancat:~/pp-ocrv3$ ls
12.jpg  ch_ppocr_mobile_v2.0_cls_infer  ch_PP-OCRv3_det_infer  ch_PP-OCRv3_rec_infer  cpp  infer_demo  ppocr_keys_v1.txt  python  visualized_result.jpg

# 推理测试，infer_demo程序的前面三个参数是model路径，第四个是字典文件，第五个是测试图片路径，最后一个参数1指定npu推理，0是指定cpu推理，详细可以看下infer_demo.cc源码
cat@lubancat:~/pp-ocrv3$ ./infer_demo
Usage: infer_demo path/to/det_model path/to/cls_model path/to/rec_model path/to/rec_label_file path/to/image run_option,
e.g ./infer_demo ./ch_PP-OCRv3_det_infer ./ch_ppocr_mobile_v2.0_cls_infer ./ch_PP-OCRv3_rec_infer ./ppocr_keys_v1.txt ./12.jpg 0
The data type of run_option is int, 0: run with cpu; 1: run with ascend.

执行cpu推理：

# CPU推理
cat@lubancat:~/pp-ocrv3$ ./infer_demo ./ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det_infer.onnx \
                      ./ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.onnx \
                      ./ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer.onnx \
                      ./ppocr_keys_v1.txt \
                      ./11.jpg \
                      0
ONNX Model
[INFO] fastdeploy/runtime/runtime.cc(326)::CreateOrtBackend     Runtime initialized with Backend::ORT in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(326)::CreateOrtBackend     Runtime initialized with Backend::ORT in Device::CPU.
[INFO] fastdeploy/runtime/runtime.cc(326)::CreateOrtBackend     Runtime initialized with Backend::ORT in Device::CPU.
det boxes: [[28,39],[301,40],[301,70],[28,69]]rec text: 纯臻营养护发素 rec score:0.939997 cls label: 0 cls score: 1.000000
det boxes: [[27,82],[171,82],[171,102],[27,102]]rec text: 产品信息/参数 rec score:0.880295 cls label: 0 cls score: 1.000000
det boxes: [[28,114],[330,114],[330,132],[28,132]]rec text: （45元/每公斤，100公斤起订） rec score:0.900614 cls label: 0 cls score: 1.000000
det boxes: [[26,143],[281,144],[281,163],[26,162]]rec text: 每瓶22元，1000瓶起订） rec score:0.930227 cls label: 0 cls score: 1.000000
det boxes: [[25,178],[300,178],[300,194],[25,194]]rec text: 【品牌】：代加工方式/OEMODM rec score:0.920930 cls label: 0 cls score: 0.990245
det boxes: [[26,210],[233,210],[233,226],[26,226]]rec text: （品名】：纯臻营养护发素 rec score:0.954969 cls label: 0 cls score: 1.000000
det boxes: [[25,241],[241,240],[241,257],[25,258]]rec text: 【产品编号】：YM-X-3011 rec score:0.880549 cls label: 0 cls score: 0.999998
det boxes: [[414,235],[428,235],[428,302],[414,302]]rec text: ODM OEM rec score:0.826402 cls label: 0 cls score: 1.000000
det boxes: [[25,272],[179,270],[179,286],[25,288]]rec text: 【净含量】：220ml rec score:0.889676 cls label: 0 cls score: 1.000000
det boxes: [[26,304],[251,304],[251,319],[26,319]]rec text: （适用人群】：适合所有肤质 rec score:0.918905 cls label: 0 cls score: 1.000000
det boxes: [[25,333],[343,334],[343,352],[25,351]]rec text: 【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚 rec score:0.934403 cls label: 0 cls score: 1.000000
det boxes: [[27,366],[280,366],[280,381],[27,381]]rec text: 糖、椰油酰胺丙基甜菜碱、泛酮 rec score:0.946751 cls label: 0 cls score: 0.999985
det boxes: [[369,369],[474,369],[474,386],[369,386]]rec text: （成品包材） rec score:0.958139 cls label: 0 cls score: 1.000000
det boxes: [[25,396],[360,395],[360,413],[25,414]]rec text: 【主要功能】：可紧致头发磷层，从而达到 rec score:0.926927 cls label: 0 cls score: 0.999998
det boxes: [[28,429],[370,429],[370,444],[28,444]]rec text: 即时持久改善头发光泽的效果，给于燥的头 rec score:0.924987 cls label: 0 cls score: 0.999634
det boxes: [[29,459],[134,459],[134,476],[29,476]]rec text: 发足够的滋养 rec score:0.994628 cls label: 0 cls score: 1.000000

Visualized result saved in ./vis_result.jpg

执行NPU推理：

# NPU推理，测试平台是lubancat-4，都是指定rk3588平台的模型文件，如果是lubancat-0/1/2使用rk3566/rk3568平台的模型
cat@lubancat:~/pp-ocrv3$ ./infer_demo ./ch_PP-OCRv3_det_infer/ch_PP-OCRv3_det_infer_rk3588_unquantized.rknn \
                          ./ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v20_cls_infer_rk3588_unquantized.rknn \
                          ./ch_PP-OCRv3_rec_infer/ch_PP-OCRv3_rec_infer_rk3588_unquantized.rknn \
                          ./ppocr_keys_v1.txt \
                          ./11.jpg \
                          1
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.0 (e6fe0c678@2023-05-25T08:09:20)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.8
index=0, name=x, n_dims=4, dims=[1, 960, 960, 3], n_elems=2764800, size=5529600, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=sigmoid_0.tmp_0, n_dims=4, dims=[1, 1, 960, 960], n_elems=921600, size=1843200, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend  Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.0 (e6fe0c678@2023-05-25T08:09:20)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.8
index=0, name=x, n_dims=4, dims=[1, 48, 192, 3], n_elems=27648, size=55296, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_0.tmp_0, n_dims=2, dims=[1, 2, 0, 0], n_elems=2, size=4, fmt=UNDEFINED, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend  Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.0 (e6fe0c678@2023-05-25T08:09:20)
[INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.8.8
index=0, name=x, n_dims=4, dims=[1, 48, 320, 3], n_elems=46080, size=92160, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
index=0, name=softmax_5.tmp_0, n_dims=4, dims=[1, 40, 6625, 1], n_elems=265000, size=530000, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0
[INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend  Runtime initialized with Backend::RKNPU2 in Device::RKNPU.
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory       The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory       The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
[WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory       The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8
det boxes: [[28,39],[301,40],[301,70],[28,69]]rec text: 纯臻营养护发素 rec score:0.938755 cls label: 0 cls score: 1.000000
det boxes: [[27,82],[171,82],[171,102],[27,102]]rec text: 产品信息/参数 rec score:0.880790 cls label: 0 cls score: 1.000000
det boxes: [[28,114],[330,114],[330,132],[28,132]]rec text: （45元/每公斤，100公斤起订） rec score:0.900850 cls label: 0 cls score: 1.000000
det boxes: [[26,143],[281,144],[281,163],[26,162]]rec text: 每瓶22元，1000瓶起订） rec score:0.929862 cls label: 0 cls score: 1.000000
det boxes: [[25,178],[300,177],[300,195],[25,195]]rec text: 【品牌】：代加工方式/OEMODM rec score:0.951919 cls label: 0 cls score: 0.997559
det boxes: [[26,210],[233,210],[233,226],[26,226]]rec text: （品名】：纯臻营养护发素 rec score:0.954630 cls label: 0 cls score: 1.000000
det boxes: [[25,241],[241,240],[241,257],[25,258]]rec text: 【产品编号】：YM-X-3011 rec score:0.879333 cls label: 0 cls score: 1.000000
det boxes: [[414,235],[428,235],[428,302],[414,302]]rec text: ODM OEM rec score:0.827253 cls label: 0 cls score: 1.000000
det boxes: [[25,272],[179,270],[179,286],[25,288]]rec text: 【净含量】：220ml rec score:0.889737 cls label: 0 cls score: 1.000000
det boxes: [[26,304],[251,304],[251,319],[26,319]]rec text: （适用人群】：适合所有肤质 rec score:0.919283 cls label: 0 cls score: 1.000000
det boxes: [[25,333],[343,334],[343,352],[25,351]]rec text: 【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚 rec score:0.934288 cls label: 0 cls score: 1.000000
det boxes: [[27,366],[280,366],[280,381],[27,381]]rec text: 糖、椰油酰胺丙基甜菜碱、泛酮 rec score:0.946010 cls label: 0 cls score: 1.000000
det boxes: [[369,369],[474,369],[474,386],[369,386]]rec text: （成品包材） rec score:0.957926 cls label: 0 cls score: 1.000000
det boxes: [[25,396],[360,395],[360,413],[25,414]]rec text: 【主要功能】：可紧致头发磷层，从而达到 rec score:0.925807 cls label: 0 cls score: 1.000000
det boxes: [[28,429],[370,429],[370,444],[28,444]]rec text: 即时持久改善头发光泽的效果，给于燥的头 rec score:0.924368 cls label: 0 cls score: 0.999512
det boxes: [[29,459],[134,459],[134,476],[29,476]]rec text: 发足够的滋养 rec score:0.994629 cls label: 0 cls score: 1.000000

Visualized result saved in ./vis_result.jpg

推理结果图片vis_result.jpg：

2.6. 参考链接

https://github.com/PaddlePaddle/FastDeploy/tree/develop/examples/vision/ocr/PP-OCR/rockchip

https://github.com/PaddlePaddle/PaddleOCR

https://github.com/rockchip-linux/rknn-toolkit2