1. TDL SDK¶
Cvitek所提供的TDL(Turnkey Deep Learning)是一个集成算法开发包,提供了便捷的编程接口, 使开发者们能够更快、高效地将各种智能技术应用于自己的产品和服务中。
目前TDL SDK包含移动侦测,人脸检测,人脸识别,人脸追踪,行人检测,语义分割,车牌辨识,车牌检测, 活体识别,IR活体识别,婴儿检测,哭声检测,姿态检测,手势侦测,手势识别等算法。
提示
教程测试LubanCat-P1板卡使用riscv64系统(其他系统类似)。
1.1. 获取tdl-sdk源码¶
tdl-sdk源码可以从LubanCat-sg200x板卡的 SDK仓库 获取,在SDK的tdl-sdk目录中。
git clone https://gitee.com/LubanCat/sophon-image-build.git --depth=1
1.2. 编译tdl-sdk¶
在SDK目录下,下载交叉编译器,然后导出环境变量。
cd sophon-image-build
# 在SDK目录下下载交叉编译工具并解压
# wget https://sophon-file.sophon.cn/sophon-prod-s3/drive/23/03/07/16/host-tools.tar.gz
# tar xvf host-tools.tar.gz
git clone https://github.com/sophgo/host-tools --depth=1
然后安装编译SDK所需的软件或者工具, 具体可以请参考下 《嵌入式Linux镜像构建与部署—LubanCat-sg200x系列板卡》 中SDK 编译选项。
# 安装软件或者工具
sudo apt install pkg-config build-essential ninja-build automake autoconf libtool wget curl git gcc g++ \
libssl-dev bc slib squashfs-tools jq cmake python3-distutils zlib1g-dev aptitude u-boot-tools traceroute \
tclsh scons parallel ssh-client tree python3-dev python3-pip device-tree-compiler libssl-dev ssh cpio dosfstools \
squashfs-tools fakeroot libncurses5 libncurses5-dev flex bison tar qemu-user-static vim p7zip-full rsync unzip mtools
编译tdl-sdk例程,需要先编译相关依赖,包括内核、osdrv、cvi_mpi等,建议直接整体编译:
source build/cvisetup.sh
# 教程测试riscv系统
export TPU_REL=1
defconfig sg2000_lubancat_riscv_ubuntu_sd
build_all
编译完成后,在install/soc_xxxx/tpu_musl_riscv64/cvitek_ai_sdk/bin目录下会生成tdl-sdk测试例程, 复制这些例程和库到系统,或者直接烧录编译生成的镜像。
1.3. 例程测试¶
测试直接烧录的最新镜像,系统目录/mnt/system/usr/bin/ai中的例程,就是编译出的测试例程:
root@lubancat:/mnt/system/usr/bin/ai# ls
sample_adas sample_clip sample_stream_consumer_counting sample_stream_hardhat sample_vi_od sample_yolov6
sample_aud_cls_read sample_lib_cvimd sample_stream_face_capture sample_stream_irregular_personvehicle_cross sample_yolo sample_yolov7
sample_aud_order sample_lib_draw_rect sample_stream_face_mask_det sample_stream_person_capture sample_yolov10 sample_yolov8
sample_cap_face sample_lib_preprocess sample_stream_face_recognition sample_stream_personvehicle_cross sample_yolov5 sample_yolox
sample_cap_face_pet sample_ppyoloe sample_stream_fr sample_vi_fd sample_yolov5_roi
这里测试sample_vi_fd人脸检测模型,先连接mipi摄像头(教程测试是gc4653),获取scrfd_768_432_int8_1x.cvimodel模型文件,然后执行下面命令:
root@lubancat:/mnt/system/usr/bin/ai# ./sample_vi_fd /home/cat/scrfd_768_432_int8_1x.cvimodel
[SAMPLE_COMM_SNS_ParseIni]-2168: Parse /mnt/data/sensor_cfg.ini
[parse_source_devnum]-1761: devNum = 1
[parse_sensor_name]-1842: sensor = GCORE_GC4653_SLAVE_MIPI_4M_30FPS_10BIT
[parse_sensor_mipidev]-1893: mipi_dev = 0
[parse_sensor_busid]-1871: bus_id = 1
[parse_sensor_i2caddr]-1882: sns_i2c_addr = 29
[parse_sensor_laneid]-1904: Lane_id = 5,3,4,-1,-1
[parse_sensor_pnswap]-1915: pn_swap = 0, 0, 0, 0, 0
[parse_sensor_mclken]-1937: mclk_en = 1
[parse_sensor_mclk]-1948: mclk = 1
MMF Version:sg200x-20241102-29-g0cc0ad7b7-64bit
Create VBPool[0], size: (5529600 * 5) = 27648000 bytes
Create VBPool[1], size: (1382400 * 5) = 6912000 bytes
Create VBPool[2], size: (2764800 * 3) = 8294400 bytes
Total memory of VB pool: 42854400 bytes
Initialize SYS and VB
# 省略..................
venc frame size: 1280x720
Initialize RTSP
rtsp://192.168.103.181/h264
prio:0
anchor:-8,-8,8,8
anchor:-16,-16,16,16
bbox:bbox_8_Conv_dequant
landmark:kps_8_Conv_dequant
score:score_8_Sigmoid_dequant
anchor:-32,-32,32,32
anchor:-64,-64,64,64
bbox:bbox_16_Conv_dequant
landmark:kps_16_Conv_dequant
score:score_16_Sigmoid_dequant
anchor:-128,-128,128,128
anchor:-256,-256,256,256
bbox:bbox_32_Conv_dequant
landmark:kps_32_Conv_dequant
score:score_32_Sigmoid_dequant
Enter TDL thread
Enter encoder thread
0 R:1436 B:3510 CT:2716
1 R:1710 B:2668 CT:3835
2 R:2578 B:1598 CT:7284
Golden 1710 1024 2668
wdrLEOnly:1
face count: 0
face count: 0
face count: 0
face count: 1
face count: 1
face count: 1
执行成功后,我们在PC上打开 VLC media player (注意板卡连接了网线,PC可以ping通开发板), 打开VLC播放器,菜单“媒体”中选择“打开网络串流”,选择“网络”标签, 在“请输入网络URL”中输入前面执行例程输出的RTSP:rtsp://192.168.103.181/h264,最后点击“播放”,就可以看到摄像头实时检测人脸的画面。