5. RKNPU2

RKNPU2 为带有RKNPU的芯片平台提供的一套跨平台的编程接口(C/C++)， 能够帮助用户部署RKNN模型或者进行矩阵乘法任务。

在使用RKNN API之前，首先需要使用RKNN-Toolkit2工具将用户的模型转换为RKNN模型（ 注意RKNN-Toolkit2，RKNPU2不同版本号可能不兼容，请选择相同的版本号 ）， 得到RKNN模型后，可以选择使用RKNN API接口在平台开发应用。

本章将简单介绍下RKNN API库的开发流程，以及使用示例。

重要

测试环境：鲁班猫RK系列板卡，镜像系统是Debian或者ubuntu，PC环境使用ubuntu20.04, 教程编写时的RKNN-Toolkit2是2.3.2。

5.1. RKNN API说明

RKNPU运行时库支持相关RK和RV系列硬件平台，对于RK3562、RK3566系列、RK3568系列、RK3576系列、RK3588系列、 RV1126B硬件平台使用rknpu2/runtime/Linux目录下的librknnrt.so（如果是Android系统就切换到rknpu2/runtime/Android目录下）， 对于RV1106/1103硬件平台使用rknpu2/runtime目录下的librknnmrt.so。

各个硬件平台的C API支持情况请参考： 04_Rockchip_RKNPU_API_Reference_RKNNRT_V2.3.2_CN 。

鲁班板卡上提供的镜像（Debian10/11，ubuntu20.04/22.04）都已经安装RKNPU运行时库，如果需要更新， 可以从 rknpu2工程 中的rknpu2/runtime/目录下获取最新runtime库。

RKNN模型部署接口分为通用API接口和零拷贝流程的API接口（rv1106/1103支持零拷贝流程的API接口,rk3562/rk356X/rk3576/rk3588系列硬件支持两组接口）。

5.1.1. 通用API接口

RKNN 通用API接口，对数据的量化，归一化和转换等等操作均在CPU，模型推理运行在NPU。其流程如下（图片参考rknpu api用户指导手册）：

相关函数介绍：

• rknn_init 接口初始化模型。

• rknn_query 函数能够查询获取到模型输入输出信息、逐层运行时间、模型推理的总时间、SDK 版本、内存占用信息、用户自定义字符串等信息

• rknn_inputs_set 函数可以设置模型的输入数据，支持多个输入。每个输入是一个rknn_input 结构体对象，在传入之前用户需要设置输入数据

• rknn_run 函数将执行一次模型推理调用，调用之前需要rknn_inputs_set函数

• rknn_outputs_get 函数可以获取模型推理的输出数据，可以获取多个输出数据

• rknn_outputs_release 函数将释放rknn_outputs_get函数得到的输出的相关资源

• rknn_destroy 函数将释放rknn_context及其相关资源。

一次简单的通用API接口流调用流程： rknn_init 函数初始化rknn_context对象和加载RKNN模型，然后使用 rknn_query 函数查询模型的输入输出信息，给后面函数使用。 使用 rknn_inputs_set 设置模型输入，调用rknn_run函数执行推理，一次推理结束后使用 rknn_outputs_get 函数获取输出数据，用户可以进行相关后处理， 处理完成后使用函数 rknn_outputs_release 释放输出相关资源，更新帧数据重复前面rknn_run函数执行推理，全部推理完成后， 最后使用 rknn_destroy 释放rknn_context及其相关资源。

建议实践下后面rknn_yolov5_demo测试例程，查看源码调用流程，这样会更清楚通用API接口的调用流程。

5.1.2. 零拷贝API接口

除了通用API接口，还有一组零拷贝API接口，它优化了对通用API的数据处理流程，数据的量化，归一化和转换等等操作都会在NPU，输出数据排布和反量化在CPU或者NPU。

两组API的主要区别在于，通用接口每次更新帧数据，需要将外部模块分配的数据拷贝到NPU运行时的输入内存，而 零拷贝流程的接口会直接使用预先分配的内存，这减少了内存拷贝的花销。零拷贝API接口调用流程和通用API接口类似， 只是设置模型的输入输出数据内存时，需要调用rknn_create_mem/_from_phys/from_fd、rknn_set_io_mem、rknn_destroy_mem等接口。

两个API接口的数据处理流程如下（参考RKNN_API指导手册）：

而实际调用情况可能比较复杂，根据是否需要自行分配模型的模块内存（输入/输出/权重/中间结果）和内存表示方式（文件描述符/物理地址等）差异， 可以分为三种典型的零拷贝调用流程：

• 输入/输出内存由运行时分配

• 输入/输出内存由外部分配

• 输入/输出/权重/中间结果内存由外部分配

详细的调用流程参考下RKNPU的用户手册。

更多RKNN API平台支持情况和接口的详细说明参考下rknpu2工程文档 02_Rockchip_RKNPU_User_Guide_RKNN_SDK_V2.3.2_CN.pdf ， 以及库头文件等等。

5.2. Linux平台测试

5.2.1. rknn_yolov5_demo例程测试

该例程使用通用API接口，具体测试步骤如下：

1、获取测试例程，使用rknpu2的rknn_yolov5_demo例程：

# 或者使用官方rknn-toolkit2
git clone https://github.com/airockchip/rknn-toolkit2.git  --depth 1
cd rknn-toolkit2/rknpu2/examples/rknn_yolov5_demo

2、 更新测试模型，可以使用前面RKNN Toolkit2转换出的yolov5 RKNN模型，也可以直接使用例程的模型。

3、 执行shell脚本，编译安装例程，可以选择在PC环境中交叉编译或者在板卡上编译。

• 在PC虚拟机上交叉编译，需要先获取交叉编译器，设置环境变量，然后编译安装，最后通过scp、rsync等方式复制install目录下的文件到板卡上。

# 安装默认交叉编译器
sudo apt update
sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu

# 或者使用下面的交叉编译器
# git clone https://github.com/LubanCat/gcc-buildroot-9.3.0-2020.03-x86_64_aarch64-rockchip-linux-gnu.git

# 切换到例程目录，如果使用从github获取的交叉编译器，需要修改下脚本文件中的GCC_COMPILER路径，或者直接export交叉编译器路径
export GCC_COMPILER=aarch64-linux-gnu

cd lubancat_ai_manual_code/dev_env/rknpu2/rknn_yolov5_demo
# 可执行权限
chmod +x build-linux.sh

# 程序指令结构
./build-linux.sh -t <target> -a <arch> [-b <build_type>]

# 根据板卡处理器，运行对应脚本,这里测试lubancat-5（处理器是rk3588），设置-t参数rk3588 ，
#  Lubancat-0/1设置rk3588 ， Lubancat-2设置rk3568 ， Lubancat-3设置rk35676
./build-linux.sh -t rk3588 -a aarch64 -b Release
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /home/llh/rknn/test/3rdparty/opencv/opencv-linux-aarch64 (found version "3.4.5")
-- Configuring done (11.8s)
-- Generating done (0.2s)
-- Build files have been written to: /home/llh/rknn/test/rknn_yolov5_demo/build/build_linux_aarch64
[ 11%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/postprocess.cc.o
[ 22%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_decoder.cpp.o
[ 33%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/main_video.cc.o
[ 44%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/main.cc.o
[ 66%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/drawing.cpp.o
[ 77%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/postprocess.cc.o
[ 88%] Linking CXX executable rknn_yolov5_video_demo
[ 88%] Built target rknn_yolov5_video_demo
[100%] Linking CXX executable rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
[ 33%] Built target rknn_yolov5_demo
[100%] Built target rknn_yolov5_video_demo
Install the project...
# 省略......
/home/llh/rknn/test/rknn_yolov5_demo

# 编译完成后，会安装程序到install/目录下,复制该文件夹内容到板卡,这里使用rsync命令同步到板卡:
rsync -avz ./install cat@192.168.103.131:~/

• 在板卡上编译类似，不同的板卡设置不同参数 。

export GCC_COMPILER=aarch64-linux-gnu

# 根据板卡处理器，运行对应脚本,这里以LubanCat-5（处理器是rk3588,板卡系统是Ubuntu24.04）为例：
cat@lubancat:~/rknpu2/rknn_yolov5_demo$ ./build-linux.sh -t rk3588 -a aarch64 -b Release
./build-linux.sh -t rk3588 -a aarch64 -b Release
aarch64-linux-gnu
===================================
TARGET_SOC=RK3588
TARGET_ARCH=aarch64
BUILD_TYPE=Release
BUILD_DIR=/xxx/rknpu2/examples/rknn_yolov5_demo/build/build_RK3588_linux_aarch64_Release
CC=aarch64-linux-gnu-gcc
CXX=aarch64-linux-gnu-g++
===================================
-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
# 省略................
-- Generating done (0.0s)
-- Build files have been written to: /xxx/rknpu2/examples/rknn_yolov5_demo/build/build_RK3588_linux_aarch64_Release
[ 10%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/postprocess.cc.o
[ 30%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/preprocess.cc.o
[ 30%] Building CXX object CMakeFiles/rknn_yolov5_demo.dir/src/main.cc.o
[ 40%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/main_video.cc.o
[ 50%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/postprocess.cc.o
[ 60%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_decoder.cpp.o
[ 70%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_encoder.cpp.o
[ 80%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/drawing.cpp.o
[ 90%] Linking CXX executable rknn_yolov5_video_demo
[ 90%] Built target rknn_yolov5_video_demo
[100%] Linking CXX executable rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
[ 40%] Built target rknn_yolov5_demo
[100%] Built target rknn_yolov5_video_demo
# 省略.....

# 编译完成后，会安装程序到install/rknn_yolov5_demo_Linux/目录下,
# 会有两个可执行文件，一个是对图片识别，一个是对视频。

在板卡上编译时RKNN API库在3rdparty目录下,运行时可以使用3rdparty文件中的RKNN API库， 也可以使用鲁班猫系统默认安装的RKNN API库。

4、测试例程

不管是交叉编译还是板卡上编译安装的install目录，进入install/rknn_yolov5_demo_Linux/目录下， 对图片进行识别，执行rknn_yolov5_demo程序，可执行文件使用命令：

# 切换到下载安装目录，运行时使用例程安装的库（一些系统可能不需要）
cat@lubancat:~/install/rknn_yolov5_demo_Linux$ export LD_LIBRARY_PATH=./lib

# 命令用法,根据板卡选择对应的模型
./rknn_yolov5_demo <rknn model> <jpg>

# 使用默认模型和图片，执行命令(测试lubuncat-5板卡，编译方式请参考前面)：
cat@lubancat:~/xxx/install/rknn_yolov5_demo_Linux$ ./rknn_yolov5_demo ./model/RK3588/yolov5s-640-640.rknn ./model/bus.jpg
post process config: box_conf_threshold = 0.25, nms_threshold = 0.45
Loading mode...
sdk version: 2.3.2 (429f97ae6b@2025-04-09T09:09:27) driver version: 0.9.8
model input num: 1, output num: 3
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, w_stride = 640, size_with_stride=1228800,
 fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=0, name=output0, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, w_stride = 0, size_with_stride=1638400,
fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=1, name=286, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, w_stride = 0, size_with_stride=491520,
 fmt=NCHW,type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=2, name=288, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, w_stride = 0, size_with_stride=163840,
 fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
model is NHWC input fmt
model input height=640, width=640, channel=3
Read ./model/bus.jpg ...
img width = 640, img height = 640
once run use 23.895000 ms
loadLabelName ./model/coco_80_labels_list.txt
person @ (209 243 286 510) 0.879723
person @ (479 238 560 526) 0.870588
person @ (109 238 231 534) 0.839831
bus @ (91 129 555 464) 0.692042
person @ (79 353 121 517) 0.300961
save detect result to ./out.jpg
loop count = 10 , average run  16.776500 ms

# 最后输出out.jpg图片，可以在板卡系统或者传输到PC查看。

对本地视频进行识别，执行rknn_yolov5_video_demo程序：

# 命令用法
./rknn_yolov5_video_demo <rknn_model> <video_path> <video_type 264/265>

# 获取视频，然后转换为h264，h265码流视频，使用命令：
# 转换成h264码流视频
ffmpeg -i xxx.mp4 -vcodec h264 out.h264

# 转换成h265码流视频
ffmpeg -i xxx.mp4 -vcodec hevc out.hevc

# 使用默认模型和视频，执行命令（教程测试Lubancat-5，使用默认的test.h264视频）：
cat@lubancat:~/xxx/install/rknn_yolov5_demo_Linux$ ./rknn_yolov5_video_demo model/RK3588/yolov5s-640-640.rknn ../../model/test.h264  264
Loading mode...
sdk version: 2.3.2 (429f97ae6b@2025-04-09T09:09:27) driver version: 0.9.8
model input num: 1, output num: 3
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=0, name=output0, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=1, name=286, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=2, name=288, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
model is NHWC input fmt
model input height=640, width=640, channel=3
mpi_dec_test start mpi_dec_test decoder test start mpp_type 7 app_ctx=0xfffff1586690 decoder=0xaaaad016efe0
read video size=4473796
receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192
 receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192 receive packet size=8192
# 省略..........
input image 1920x1080 stride 1920x1088 format=2560
resize with RGA!
rga_api version 1.10.1_[0]
once run use 32.529000 ms
post process config: box_conf_threshold = 0.25, nms_threshold = 0.45
loadLabelName ./model/coco_80_labels_list.txt
sheep @ (615 934 717 987) 0.662699
bird @ (279 999 342 1036) 0.361400
bird @ (0 889 168 1029) 0.356540
chn 0  size 412449  qp 17
time_gap=-25decoder require buffer w:h [1920:1080] stride [1920:1088] buf_size 4177920
pts=0 dts=0 get one frame 1789194765699 data_vir=0xffffad261000 fd=35 input image 1920x1080 stride 1920x1088 format=2560
# 省略.......
time_gap=7found last frame reset decoder
waiting finish

# 最后会在当前目录下生成out.h264视频，可以在板卡系统上播放视频查看结果。

如果运行程序出现：

mpi_dec_test start mpi_dec_test decoder test start mpp_type 7 0xaaab0b901620 mpp_init failed app_ctx=0xffffd83adb70 decoder=0xaaab0b903fe0
read video size=4473796
Segmentation fault (core dumped)

可能是没有权限导致，执行程序命令加上sudo，或者通过udev等方式修改设备文件 /dev/mpp_service 的权限。

对rtsp视频流简单测试，也是使用rknn_yolov5_video_demo程序,教程测试win上运行的 mediamtx 作为服务器， 使用 ffmpeg 推流，然后板卡（Lubancat-5）执行程序拉流检测。

# 从点击前面mediamtx链接，，获取压缩包，解压后打开powershell执行mediamtx.exe，启动后默认监听：RTSP：8554端口
.\mediamtx.exe
2026/09/12 15:44:09 INF MediaMTX v1.6.0
2026/09/12 15:44:09 INF configuration loaded from E:\media\rtsp\mediamtx.yml
2026/09/12 15:44:09 INF [RTSP] listener opened on :8554 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
2026/09/12 15:44:09 INF [RTMP] listener opened on :1935
2026/09/12 15:44:09 INF [HLS] listener opened on :8888
2026/09/12 15:44:09 INF [WebRTC] listener opened on :8889 (HTTP), :8189 (ICE/UDP)
2026/09/12 15:44:09 INF [SRT] listener opened on :8890 (UDP)


# 当前前面ffmpeg链接，获取安装包安装
# ffmpeg推流测试，输出到rtsp://localhost:8554/live/stream，使用测试视频test.mp4（从教程网盘获取）,注意修改路径
.\ffmpeg.exe -re -i "./test.mp4" -vcodec h264 -f rtsp -rtsp_transport tcp rtsp://192.168.103.104:8554/live/stream

# 循环推流
# .\ffmpeg.exe -re -stream_loop -1 -i "./test.mp4"  -c copy -f rtsp -rtsp_transport tcp rtsp://192.168.103.104:8554/live/stream

确认板卡网络和win主机网络在同一网段下，然后测试拉流（ffmpeg非循环推流）：

# 命令用法
./rknn_yolov5_video_demo model/<TARGET_PLATFORM>/yolov5s-640-640.rknn  <RTSP_URL> 264

# 板卡上执行命令，测试使用前面本地推流的视频rtsp://127.0.0.1:8554/stream，仅仅是测试
./rknn_yolov5_video_demo ./model/RK3588/yolov5s-640-640.rknn rtsp://192.168.103.104:8554/live/stream 264
Loading mode...
sdk version: 2.3.2 (429f97ae6b@2025-04-09T09:09:27) driver version: 0.9.8
model input num: 1, output num: 3
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=0, name=output0, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=1, name=286, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=2, name=288, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
model is NHWC input fmt
model input height=640, width=640, channel=3
mpi_dec_test start mpi_dec_test decoder test start mpp_type 7 app_ctx=0xffffd7fe0d40 decoder=0xaaab137fefe0
2026-09-12 15:57:35.684 D rknn_yolov5_video_demo[275532-stamp thread] util.cpp:366 operator() | Stamp thread started
enter any key to exit
2026-09-12 15:57:35.686 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:466 EventPollerPool | EventPoller created size: 8
2026-09-12 15:57:35.686 D rknn_yolov5_video_demo[275532-event poller 0] RtspPlayer.cpp:93 play | rtsp://192.168.103.104:8554/live/stream null null 0
2026-09-12 15:57:35.699 W rknn_yolov5_video_demo[275532-event poller 0] RtspPlayer.cpp:689 onPlayResult_l | 0 rtsp play success
play success!on_track_frame_out ctx=0xffffd7fe0d40
decoder=0xaaab137fefe0
receive packet size=4291 2026-09-12 15:57:35.749 D rknn_yolov5_video_demo[275532-event poller 0]
MediaSink.cpp:137 emitAllTrackReady | all track ready use 58ms
2026-09-12 15:57:35.749 I rknn_yolov5_video_demo[275532-event poller 0] main_video.cc:505 on_mk_play_event_func | got video track: H264
on_track_frame_out ctx=0xffffd7fe0d40
decoder=0xaaab137fefe0
receive packet size=60074 on_track_frame_out ctx=0xffffd7fe0d40
decoder=0xaaab137fefe0
receive packet size=6536 on_track_frame_out ctx=0xffffd7fe0d40
decoder=0xaaab137fefe0
receive packet size=1481 on_track_frame_out ctx=0xffffd7fe0d40
decoder=0xaaab137fefe0

# 省略.......
bird @ (576 246 1062 904) 0.757401
chn 0  size 57119   qp 11
time_gap=5decoder require buffer w:h [1920:1080] stride [1920:1088] buf_size 4177920 pts=0 dts=0 get one frame
1789199864654 data_vir=0xffff719f6000 fd=51 input image 1920x1080 stride 1920x1088 format=2560
resize with RGA!
once run use 18.385000 ms
post process config: box_conf_threshold = 0.25, nms_threshold = 0.45
person @ (378 216 1248 896) 0.485813
chn 0  size 217838  qp 11
time_gap=3on_track_frame_out ctx=0xffffd7fe0d40
decoder=0xaaab137fefe0
receive packet size=11889 on_track_frame_out ctx=0xffffd7fe0d40
decoder=0xaaab137fefe0
receive packet size=7969 play interrupted: 1 end of file2026-09-12 15:57:44.687 W rknn_yolov5_video_demo[275532-event poller 0]
RtspPlayer.cpp:689 onPlayResult_l | 1 end of file
2026-09-12 15:57:49.693 W rknn_yolov5_video_demo[275532-event poller 0] RtspPlayer.cpp:689 onPlayResult_l | 2 receive rtp timeout

waiting finish
2026-09-12 15:58:12.611 D rknn_yolov5_video_demo[275532-rknn_yolov5_vid] RtspPlayerImp.h:31 ~RtspPlayerImp |
2026-09-12 15:58:12.612 D rknn_yolov5_video_demo[275532-rknn_yolov5_vid] RtspPlayer.cpp:41 ~RtspPlayer |
2026-09-12 15:58:15.630 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab137eb210
2026-09-12 15:58:15.631 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab13e18450
2026-09-12 15:58:15.631 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab13e18a50
2026-09-12 15:58:15.632 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab13e19080
2026-09-12 15:58:15.632 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab13e19680
2026-09-12 15:58:15.632 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab13e19c80
2026-09-12 15:58:15.632 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab13e1a230
2026-09-12 15:58:15.632 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] EventPoller.cpp:98 ~EventPoller | 0xaaab13e1a830
2026-09-12 15:58:15.633 I rknn_yolov5_video_demo[275532-rknn_yolov5_vid] logger.cpp:86 ~Logger |

按任意键退出，最后会在当前目录下生成out.h264视频，可以在板卡系统上播放视频查看结果。

5.2.2. 零拷贝API接口例程

测试例程可以从 RKNPU工程文件 中examples/rknn_api_demo_Linux目录获取，或者从教程代码例程获取。 该例程使用MobileNet-v1进行图像识别，在linux平台上测试有两个示例，区别是一个使用RGA，一个不使用RGA。

1、获取例程（和前面例程在同一目录下）

# 或者从rknpu2 github获取
git clone https://github.com/airockchip/rknn-toolkit2 --depth 1
cd rknn-toolkit2/rknpu2/examples/rknn_api_demo/

# 切换到例程目录下
cd  rknpu2/rknn_api_demo

2、编译例程，演示直接在板卡上编译（LubanCat-4）：

# 可执行权限
chmod +x build-linux.sh

# 教程测试Lubancat-5，命令格式：./build-linux.sh -t <target> -a <arch> -b <build_type>
cat@lubancat:~/xxx/rknpu2/rknn_api_demo$ ./build-linux.sh -t rk3588 -a aarch64 -b Release
./build-linux.sh -t rk3588 -a aarch64 -b Release
aarch64-linux-gnu
===================================
TARGET_SOC=RK3588
TARGET_ARCH=aarch64
BUILD_TYPE=Release
BUILD_DIR=/xxx/rknpu2/examples/rknn_api_demo/build/build_RK3588_linux_aarch64_Release
CC=aarch64-linux-gnu-gcc
CXX=aarch64-linux-gnu-g++
===================================
-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/aarch64-linux-gnu-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/aarch64-linux-gnu-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (0.8s)
-- Generating done (0.0s)
-- Build files have been written to: /xxx/rknpu2/examples/rknn_api_demo/build/build_RK3588_linux_aarch64_Release
[ 50%] Building CXX object CMakeFiles/rknn_create_mem_demo.dir/src/rknn_create_mem_demo.cpp.o
[ 50%] Building CXX object CMakeFiles/rknn_create_mem_with_rga_demo.dir/src/rknn_create_mem_with_rga_demo.cpp.o
[ 75%] Linking CXX executable rknn_create_mem_with_rga_demo
[100%] Linking CXX executable rknn_create_mem_demo
[100%] Built target rknn_create_mem_with_rga_demo
[100%] Built target rknn_create_mem_demo
[ 50%] Built target rknn_create_mem_demo
[100%] Built target rknn_create_mem_with_rga_demo
# 省略....

编译完成后，会安装程序在当前目录install/下。

3、测试例程

# 测试使用当前目录install/rknn_api_demo_Linux下的rknn_create_mem_demo,
# 请根据实际目标设置模型路径
cat@lubancat:~/xxx/install/rknn_api_demo_Linux$ ./rknn_create_mem_demo model/RK3588/mobilenet_v1.rknn model/dog_224x224.jpg
rknn_api/rknnrt version: 1.5.0 (e6fe0c678@2023-05-25T08:09:20), driver version: 0.8.8
model input num: 1, output num: 1
input tensors:
index=0, name=input, n_dims=4, dims=[1, 224, 224, 3], n_elems=150528, size=150528, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=0, scale=0.007812
output tensors:
index=0, name=MobilenetV1/Predictions/Reshape_1, n_dims=2, dims=[1, 1001, 0, 0], n_elems=1001,
 size=2002, fmt=UNDEFINED, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000
custom string:
Begin perf ...
0: Elapse Time = 3.15ms, FPS = 317.56
---- Top5 ----
0.878906 - 156
0.059814 - 155
0.004082 - 205
0.003651 - 284
0.000215 - 285


# 测试使用当前目录install/rknn_api_demo_Linux下的rknn_create_mem_with_rga_demo
cat@lubancat:~/xxx/install/rknn_api_demo_Linux$ ./rknn_create_mem_with_rga_demo model/RK3588/mobilenet_v1.rknn model/dog_224x224.jpg
rknn_api/rknnrt version: 2.3.2 (429f97ae6b@2025-04-09T09:09:27), driver version: 0.9.8
model input num: 1, output num: 1
input tensors:
index=0, name=input, n_dims=4, dims=[1, 224, 224, 3], n_elems=150528, size=150528, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=0, scale=0.007812
output tensors:
index=0, name=MobilenetV1/Predictions/Reshape_1, n_dims=2, dims=[1, 1001, 0, 0], n_elems=1001,
 size=2002, fmt=UNDEFINED, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000
custom string:
rga_api version 1.10.1_[0]
Begin perf ...
0: Elapse Time = 2.98ms, FPS = 335.23
---- Top5 ----
0.878906 - 156
0.059814 - 155
0.004082 - 205
0.003651 - 284
0.000215 - 285

最终输出概率最大的5个类别及其对应概率，可以看出概率最大的是类别156。测试过程出现failed to open RGA:Permission denied，请使用加sudo在命令前面。

4、零拷贝API接口例程源码，以不带rga为例（rknn_create_mem_demo.cpp），下面只列出了主函数部分：

int main(int argc, char* argv[])
{
    // 省略了部分程序...

    rknn_context ctx = 0;

    // 调用rknn_init接口初始化rknn_context，加载RKNN模型
    int            model_len = 0;
    unsigned char* model     = load_model(model_path, &model_len);
    int            ret       = rknn_init(&ctx, model, model_len, 0, NULL);
    if (ret < 0) {
        printf("rknn_init fail! ret=%d\n", ret);
        return -1;
    }

    // 调用rknn_query接口，查询查询获取SDK 版本、模型输入输出信息
    rknn_sdk_version sdk_ver;
    ret = rknn_query(ctx, RKNN_QUERY_SDK_VERSION, &sdk_ver, sizeof(sdk_ver));
    if (ret != RKNN_SUCC) {
        printf("rknn_query fail! ret=%d\n", ret);
        return -1;
    }

    printf("rknn_api/rknnrt version: %s, driver version: %s\n", sdk_ver.api_version, sdk_ver.drv_version);

    // Get Model Input Output Info
    rknn_input_output_num io_num;
    ret = rknn_query(ctx, RKNN_QUERY_IN_OUT_NUM, &io_num, sizeof(io_num));
    if (ret != RKNN_SUCC) {
        printf("rknn_query fail! ret=%d\n", ret);
        return -1;
    }
    printf("model input num: %d, output num: %d\n", io_num.n_input, io_num.n_output);

    // 调用rknn_query接口，查询原始的输入tensor属性，输出的tensor属性，放到对应rknn_tensor_attr结构体对象
    printf("input tensors:\n");
    rknn_tensor_attr input_attrs[io_num.n_input];
    memset(input_attrs, 0, io_num.n_input * sizeof(rknn_tensor_attr));
    for (uint32_t i = 0; i < io_num.n_input; i++) {
        input_attrs[i].index = i;
        // query info
        ret = rknn_query(ctx, RKNN_QUERY_INPUT_ATTR, &(input_attrs[i]), sizeof(rknn_tensor_attr));
        if (ret < 0) {
        printf("rknn_init error! ret=%d\n", ret);
        return -1;
        }
        dump_tensor_attr(&input_attrs[i]);
    }

    printf("output tensors:\n");
    rknn_tensor_attr output_attrs[io_num.n_output];
    memset(output_attrs, 0, io_num.n_output * sizeof(rknn_tensor_attr));
    for (uint32_t i = 0; i < io_num.n_output; i++) {
        output_attrs[i].index = i;
        // query info
        ret = rknn_query(ctx, RKNN_QUERY_OUTPUT_ATTR, &(output_attrs[i]), sizeof(rknn_tensor_attr));
        if (ret != RKNN_SUCC) {
        printf("rknn_query fail! ret=%d\n", ret);
        return -1;
        }
        dump_tensor_attr(&output_attrs[i]);
    }

    // 调用rknn_query接口，查询 RKNN 模型里面的用户自定义字符串信息
    rknn_custom_string custom_string;
    ret = rknn_query(ctx, RKNN_QUERY_CUSTOM_STRING, &custom_string, sizeof(custom_string));
    if (ret != RKNN_SUCC) {
        printf("rknn_query fail! ret=%d\n", ret);
        return -1;
    }
    printf("custom string: %s\n", custom_string.string);

    unsigned char*     input_data   = NULL;
    rknn_tensor_type   input_type   = RKNN_TENSOR_UINT8;
    rknn_tensor_format input_layout = RKNN_TENSOR_NHWC;

    // 加载图片，输入数据
    input_data = load_image(input_path, &input_attrs[0]);

    if (!input_data) {
        return -1;
    }

    // 使用rknn_create_mem分配内存，存放输入tensor
    rknn_tensor_mem* input_mems[1];
    // default input type is int8 (normalize and quantize need compute in outside)
    // if set uint8, will fuse normalize and quantize to npu
    input_attrs[0].type = input_type;
    // default fmt is NHWC, npu only support NHWC in zero copy mode
    input_attrs[0].fmt = input_layout;

    input_mems[0] = rknn_create_mem(ctx, input_attrs[0].size_with_stride);

    // Copy input data to input tensor memory
    int width  = input_attrs[0].dims[2];
    int stride = input_attrs[0].w_stride;

    if (width == stride) {
        memcpy(input_mems[0]->virt_addr, input_data, width * input_attrs[0].dims[1] * input_attrs[0].dims[3]);
    } else {
        int height  = input_attrs[0].dims[1];
        int channel = input_attrs[0].dims[3];
        // copy from src to dst with stride
        uint8_t* src_ptr = input_data;
        uint8_t* dst_ptr = (uint8_t*)input_mems[0]->virt_addr;
        // width-channel elements
        int src_wc_elems = width * channel;
        int dst_wc_elems = stride * channel;
        for (int h = 0; h < height; ++h) {
        memcpy(dst_ptr, src_ptr, src_wc_elems);
        src_ptr += src_wc_elems;
        dst_ptr += dst_wc_elems;
        }
    }

    // 使用rknn_create_mem函数分配内存，存放输出tensor
    rknn_tensor_mem* output_mems[io_num.n_output];
    for (uint32_t i = 0; i < io_num.n_output; ++i) {
        // default output type is depend on model, this require float32 to compute top5
        // allocate float32 output tensor
        int output_size = output_attrs[i].n_elems * sizeof(float);
        output_mems[i]  = rknn_create_mem(ctx, output_size);
    }

    // rknn_set_io_mem函数设置输入tensor内存，NPU使用
    ret = rknn_set_io_mem(ctx, input_mems[0], &input_attrs[0]);
    if (ret < 0) {
        printf("rknn_set_io_mem fail! ret=%d\n", ret);
        return -1;
    }

    // rknn_set_io_mem函数设置输出tensor内存，NPU使用
    for (uint32_t i = 0; i < io_num.n_output; ++i) {
        // default output type is depend on model, this require float32 to compute top5
        output_attrs[i].type = RKNN_TENSOR_FLOAT32;
        // set output memory and attribute
        ret = rknn_set_io_mem(ctx, output_mems[i], &output_attrs[i]);
        if (ret < 0) {
        printf("rknn_set_io_mem fail! ret=%d\n", ret);
        return -1;
        }
    }

    // 调用rknn_run接口执行一次模型推理
    printf("Begin perf ...\n");
    for (int i = 0; i < loop_count; ++i) {
        int64_t start_us  = getCurrentTimeUs();
        ret               = rknn_run(ctx, NULL);
        int64_t elapse_us = getCurrentTimeUs() - start_us;
        if (ret < 0) {
        printf("rknn run error %d\n", ret);
        return -1;
        }
        printf("%4d: Elapse Time = %.2fms, FPS = %.2f\n", i, elapse_us / 1000.f, 1000.f * 1000.f / elapse_us);
    }

    // 对推理结果处理，获取概率最大的5个类别及其对应概率，并打印输出
    uint32_t topNum = 5;
    for (uint32_t i = 0; i < io_num.n_output; i++) {
        uint32_t MaxClass[topNum];
        float    fMaxProb[topNum];
        float*   buffer    = (float*)output_mems[i]->virt_addr;
        uint32_t sz        = output_attrs[i].n_elems;
        int      top_count = sz > topNum ? topNum : sz;

        rknn_GetTopN(buffer, fMaxProb, MaxClass, sz, topNum);

        printf("---- Top%d ----\n", top_count);
        for (int j = 0; j < top_count; j++) {
        printf("%8.6f - %d\n", fMaxProb[j], MaxClass[j]);
        }
    }

    // 使用rknn_destroy_mem函数销毁输入输出的tensor内存
    rknn_destroy_mem(ctx, input_mems[0]);
    for (uint32_t i = 0; i < io_num.n_output; ++i) {
        rknn_destroy_mem(ctx, output_mems[i]);
    }

    // 释放rknn_context，以及其相关资源
    rknn_destroy(ctx);

    if (input_data != nullptr) {
        free(input_data);
    }

    if (model != nullptr) {
        free(model);
    }

    return 0;
}

5.3. Android平台测试

在安卓平台上运行RKNN模型也是类似的，下面将参考RKNPU2例程，进行简单测试。

重要

鲁班猫烧录安卓镜像请参考下 安卓教程 ， 教程测试使用lubancat-4，android 12镜像。

启动系统后，我们需要通过adb连接板卡， 具体连接的usb接口请参考 adb使用手册 的硬件说明。 下面以lubancat-4为例（鲁班猫RK系列其他板卡类似），我们使用usb-typec线连接板子usb3.0 OTG接口，另外一端连接到PC电脑：

然后在PC电脑(ubuntu系统)上使用adb连接设备：

root@YH-LONG:~# adb devices
* daemon not running; starting now at tcp:5037
* daemon started successfully
List of devices attached
7fbff6f219991594        device

5.3.1. rknn_yolov5_demo例程测试

1、拉取例程和获取编译器

# 从教程配套例程
git clone https://gitee.com/LubanCat/lubancat_ai_manual_code.git
#cd dev_env/rknpu2/rknn_yolov5_demo

# 最新的rknpu2仓库是https://github.com/airockchip/rknn-toolkit2/tree/master/rknpu2

然后从 https://developer.android.google.cn/ndk/downloads?hl=zh-cn 获取NDK工具， 教程测试rknpu2建议使用的NDK版本（android-ndk-r17c，Linux x86 64位），NDK也可以从配套 网盘资源 （提取码hslu）获取， 最终获取到的zip文件解压到相关目录下。

2、编译rknn_yolov5_demo例程

# 切换到rknpu2工程的rknpu2-master/examples/rknn_yolov5_demo目录下
cd rknpu2-master/examples/rknn_yolov5_demo

# 或者是从教程配套例程
#cd dev_env/rknpu2/rknn_yolov5_demo

# 然后修改编译脚本build-android_RK3588.sh（测试的是安卓系统，lubancat-4）中的ANDROID_NDK_PATH路径,
# 具体根据前面获取的android-ndk-r26b存放路径
    ANDROID_NDK_PATH=/mnt/d/rk/android-ndk-r17c

# 执行脚本，编译例程
./build-android_RK3588.sh
-- Check for working C compiler: /mnt/d/rk/android-ndk-r17c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang
-- Check for working C compiler: /mnt/d/rk/android-ndk-r17c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /mnt/d/rk/android-ndk-r17c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++
-- Check for working CXX compiler: /mnt/d/rk/android-ndk-r17c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /mnt/d/rk/rknpu2-1.5.2/examples/3rdparty/opencv/OpenCV-android-sdk (found version "3.4.5")
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/d/rk/rknpu2-1.5.2/examples/rknn_yolov5_demo/build/build_android_v8a
Scanning dependencies of target rknn_yolov5_video_demo
Scanning dependencies of target rknn_yolov5_demo
[ 11%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/main_video.cc.o
[ 22%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/src/postprocess.cc.o
[ 33%] Building CXX object CMakeFiles/rknn_yolov5_video_demo.dir/utils/mpp_decoder.cpp.o
# 省略...
[ 88%] Built target rknn_yolov5_video_demo
[100%] Linking CXX executable rknn_yolov5_demo
[100%] Built target rknn_yolov5_demo
[ 66%] Built target rknn_yolov5_video_demo
[100%] Built target rknn_yolov5_demo
Install the project...
# 省略...

# 编译出的程序会安装在当前目录下install中

3、传输程序到板卡

# 把前面编译好的例程传输到/data目录下
adb root
adb push install/rknn_yolov5_demo_Android /data

# 进入板卡系统终端，切换到例程目录下
adb shell
rk3588s_lubancat_4_hdmi:/# cd /data/rknn_yolov5_demo_Android
rk3588s_lubancat_4_hdmi:/data/rknn_yolov5_demo_Android#

# 然后设置库的搜索路径，使用我们传输进来的库
rk3588s_lubancat_4_hdmi:/data/rknn_yolov5_demo_Android# export  LD_LIBRARY_PATH=./lib

# 为rknn_yolov5_demo添加可执行权限，然后执行命令，命令用法是：./rknn_yolov5_demo <rknn model> <jpg>
rk3588s_lubancat_4_hdmi:/data/rknn_yolov5_demo_Android# chmod +x rknn_yolov5_demo
rk3588s_lubancat_4_hdmi:/data/rknn_yolov5_demo_Android# ./rknn_yolov5_demo ./model/RK3588/yolov5s-640-640.rknn ./model/bus.jpg
model input num: 1, output num: 3
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, w_stride = 640, size_with_stride=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE,
zp=-128, scale=0.003922
index=0, name=output, n_dims=4, dims=[1, 255, 80, 80], n_elems=1632000, size=1632000, w_stride = 0, size_with_stride=1638400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp
=-128, scale=0.003860
index=1, name=283, n_dims=4, dims=[1, 255, 40, 40], n_elems=408000, size=408000, w_stride = 0, size_with_stride=491520, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128,
scale=0.003922
index=2, name=285, n_dims=4, dims=[1, 255, 20, 20], n_elems=102000, size=102000, w_stride = 0, size_with_stride=163840, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128,
scale=0.003915
model is NHWC input fmt
model input height=640, width=640, channel=3
once run use 31.285000 ms
loadLabelName ./model/coco_80_labels_list.txt
person @ (209 244 286 506) 0.884139
person @ (478 238 559 526) 0.867678
person @ (110 238 230 534) 0.824685
bus @ (94 129 553 468) 0.705055
person @ (79 354 122 516) 0.339254
loop count = 10 , average run  29.712800 ms

# 结果保存在out.jpg中，exit命令退出终端，然后使用adb命令传输到PC本地查看
adb pull /data/rknn_yolov5_demo_Android/out.jpg ./
/data/rknn_yolov5_demo_Android/out.jpg: 1 file pulled, 0 skipped. 25.3 MB/s (190629 bytes in 0.007s)

测试运行时指定了搜索库路径，如果使用系统默认安装的库，可能需要更新板端的runtime库(替换安卓系统的/vendor/lib64/librknnrt.so),以及rknn_server。 使转换出rknn模型的RKNN-Toolkit2版本和runtime库版本一致。

5.4. 参考链接

https://github.com/rockchip-linux/rknpu2

https://github.com/airockchip/rknn-toolkit2