RDK X5部署YOLOv11,(非TROS)最高可达140FPS
特奶
2025年07月27日 22:44

RDK X5部署YOLOv11,最高可达140FPS(非TROS方式)

不懂原理,只说步骤,话不多说,直接开始

1. 下载OE开发包

代码块
JavaScript
自动换行
复制代码
wget -c ftp://x5ftp@vrftp.horizon.ai/OpenExplorer/v1.2.8_release/horizon_x5_open_explorer_v1.2.8-py310_20240926.tar.gz --ftp-password=x5ftp@123$%
复制成功

2. 下载Docker镜像

CPU和GPU二选一都可以,我这里选择下载的是GPU的镜像

代码块
JavaScript
自动换行
复制代码
# CPU
wget -c ftp://x5ftp@vrftp.horizon.ai/OpenExplorer/v1.2.8_release/docker_openexplorer_ubuntu_20_x5_cpu_v1.2.8.tar.gz --ftp-password=x5ftp@123$%
#GPU
wget -c ftp://x5ftp@vrftp.horizon.ai/OpenExplorer/v1.2.8_release/docker_openexplorer_ubuntu_20_x5_gpu_v1.2.8.tar.gz --ftp-password=x5ftp@123$%
复制成功

3. 下载ultralytics仓库

代码块
JavaScript
自动换行
复制代码
git clone https://github.com/ultralytics/ultralytics.git
cd ultralytics/ultralytics/nn/
复制成功

head.py文件中的forward()函数注释掉,新增一个同名函数(注意代码缩进)

代码块
JavaScript
自动换行
复制代码
def forward(self, x):
        bboxes = [self.cv2[i](x[i]).permute(0, 2, 3, 1).contiguous() for i in range(self.nl)]
        clses = [self.cv3[i](x[i]).permute(0, 2, 3, 1).contiguous() for i in range(self.nl)]
        return (bboxes, clses)
复制成功

此步骤主要是用于将模型转换为onnx时使用:

forward函数

4. 创建conda环境

代码块
JavaScript
自动换行
复制代码
conda create -n rdkx5 python=3.9
conda activate rdkx5
pip install ultralytics -i https://pypi.tuna.tsinghua.edu.cn/simple
复制成功

5. 训练模型

注意训练前的forward函数应该是YOLOv11原始的forward函数

注意这里

数据集自行准备,我这里默认你有了模型文件

代码块
JavaScript
自动换行
复制代码
cd ultralytics
wget https://github.com/ultralytics/assets/releases/download/v8.3.0/yolo11n.pt
复制成功

开始训练(具体参数自行了解)

代码块
JavaScript
自动换行
复制代码
yolo detect train model=/home/juice/horizon_rdkx5/ultralytics/yolo11n.pt data=data.yaml epochs=10 imgsz=640 batch=32 device=0 name=yolo11_test_run amp=True cache=True
复制成功

6. pt转onnx

转换前将forward函数切换成上文中(第3步)所说的转换onnx所用的版本

这里不要忘

默认你已经拥有了模型文件(.pt),创建export_onnx.py并且执行(这一步骤是将bin转换为onnx),如果提示有依赖缺失请自行安装,最终转换出的onnx文件和.pt文件在同一个文件夹下

代码块
JavaScript
自动换行
复制代码
from ultralytics import YOLO

def main():
    # 模型路径
    model_path = '你的模型.pt文件的路径'
    
    # 加载模型
    model = YOLO(model_path)
    
    # 导出为 ONNX 格式
    model.export(
        imgsz=640, # 输入尺寸
        format='onnx', # 导出格式
        simplify=True, # 是否简化模型(需要安装 onnxsim)
        opset=11 # ONNX opset 版本
    )

    print("导出完成!")

if __name__ == '__main__':
    main()
复制成功

7. 配置OE环境

首先加载docker镜像(压缩文件无需解压)

代码块
JavaScript
自动换行
复制代码
# 加载docker镜像,我这里加载的是GPU的(CPU同)
docker load < docker_openexplorer_ubuntu_20_x5_gpu_v1.2.8.tar.gz
复制成功

加载后输入docker images查看镜像,并且记住TAG,等下要用

记住TAG

解压OE开发包,我将解压后的文件夹命名为,看到后续步骤不要疑惑:oe_v1.2.8

代码块
JavaScript
自动换行
复制代码
tar -xvf horizon_x5_open_explorer_v1.2.8-py310_20240926.tar.gz
复制成功

进入oe_v1.2.8文件夹,编辑run_docker.sh

代码块
JavaScript
自动换行
复制代码
#!/bin/bash

dataset_path=$1
run_type=$2
version=v1.2.8-py310(这里填写你的镜像的TAG)
container_name=$(whoami)_OE_v1.2.8
host_name=$(echo "1.2.8" |awk -F "." '{ print $1"-"$2"-"$3 }')

if [ -z "$dataset_path" ];then
  echo "Please specify the dataset path"
  exit
fi
dataset_path=$(readlink -f "$dataset_path")

echo "Docker version is ${version}"
echo "Dataset path is $(readlink -f "$dataset_path")"

open_explorer_path=$(readlink -f "$(dirname "$0")")
echo "OpenExplorer package path is $open_explorer_path"

if [ "$run_type" == "cpu" ];then
    echo "Start Docker container in CPU mode."
    docker run -it --rm \
      --hostname "OE-X5-CPU-$host_name" \
      --name $container_name \
      -v "$open_explorer_path":/open_explorer \
      -v "$dataset_path":/data/horizon_x5/data \
      openexplorer/ai_toolchain_ubuntu_20_x5_cpu:"$version"
else
    echo "Start Docker container in GPU mode."
    docker run -it --rm \
      --hostname "OE-X5-GPU-$host_name" \
      --name $container_name \
      --gpus all \
      --shm-size="15g" \
      -v "$open_explorer_path":/open_explorer \
      -v "$dataset_path":/data/horizon_x5/data \
      openexplorer/ai_toolchain_ubuntu_20_x5_gpu:"$version"
fi
复制成功

编辑好后在这个文件夹中创建一个名为models的文件夹,将你要使用的onnx文件放入该文件夹

然后创建calibration_images文件夹,并且在其中放入100张训练模型时的图片

放入好图片后,创建generate_calibration_data.py,文件内容为:

代码块
JavaScript
自动换行
复制代码
#!/user/bin/env python

# Copyright (c) 2024,WuChao D-Robotics.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# 注意: 此程序推荐在工具链Docker中运行
# Attention: This program is recommended to run in OpenExplore Docker.



import os
import shutil
from time import time
import cv2
import numpy as np

import argparse
import logging 

# 日志模块配置
# logging configs
logging.basicConfig(
    level = logging.DEBUG,
    format = '[%(name)s] [%(asctime)s.%(msecs)03d] [%(levelname)s] %(message)s',
    datefmt='%H:%M:%S')
logger = logging.getLogger("RDK_YOLO")

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('--src', type=str, default='origin_coco_imgs', help="Source images path") 
    parser.add_argument('--dist', type=str, default='calibration_data_rgb_f32_640', help="Destination images path") 
    parser.add_argument('--width', type=int, default=640, help="W in ONNX NCHW.") 
    parser.add_argument('--height', type=int, default=640, help="H in ONNX NCHW.")

    opt = parser.parse_args()
    logger.info(opt)

    # 检查源图片文件夹是否存在
    if not os.path.exists(opt.src):
        logger.error("Source images path is not exist, please check!")
        exit()

    # 如果目标文件夹存在, 则删除目标文件夹
    if os.path.exists(opt.dist):
        shutil.rmtree(opt.dist)
        logger.info("Destination folder already exists, removed")
    os.makedirs(opt.dist)
    logger.info("\033[1;31m" + f"Created directory Successfully: \"{opt.dist}\"" + "\033[0m")

    # 逐个转化并保存
    begin_time = time()
    img_names = os.listdir(opt.src)
    cnt_total = len(img_names)
    for cnt, img_name in enumerate(img_names, 1):
        img_path = os.path.join(opt.src, img_name)
        img = cv2.imread(img_path)
        # 此处的前处理以ONNX的前处理为基础,总的来说是和训练时的前处理保持一致
        # 如果yaml中有配置mean和scale, 则此处无须计算mean和scale.
        input_tensor = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)     # BGR2RGB
        input_tensor = cv2.resize(img, (opt.width, opt.height)) # resize
        input_tensor = np.transpose(input_tensor, (2, 0, 1))    # HWC2CHW
        input_tensor = np.expand_dims(input_tensor, axis=0).astype(np.float32)  # CHW -> NCHW
        dst_path = os.path.join(opt.dist, img_name[:-4] + '.rgbchw') # tofile
        input_tensor.tofile(dst_path)
        logger.info(f"[\033[1;32m{cnt}\033[0m/\033[1;32m{cnt_total}\033[0m] write: {dst_path}")
    logger.info("\033[1;31m" + "The calibration data has been successfully generated, time = %.1f s"%(time() - begin_time) + "\033[0m")
if __name__ == "__main__":
    main()
复制成功

这个脚本的使用格式为,改脚本执行后会创建一个calibration_images_640(校准数据集)文件夹:

代码块
JavaScript
自动换行
复制代码
python3 generate_calib_data.py --src ./calibration_images --dist ./calibration_images_640 --width 640 --height 640
复制成功

执行好脚本后创建config_yolo11_detect_bayese_640x640_nv12.yaml,文件内容为:

代码块
JavaScript
自动换行
复制代码
model_parameters:
  onnx_model: './models/ppe/best.onnx' # docker镜像中你的onnx的路径
  march: "bayes-e"
  layer_out_dump: False
  working_dir: 'bin_dir/yolo11n_detect_bayese_640x640_nv12' # 转换为bin文件后的文件夹名
  output_model_file_prefix: 'yolo11n_detect_bayese_640x640_nv12' # 转换为bin文件的名称
  # YOLO11 n, s, m
  node_info: {"/model.10/m/m.0/attn/Softmax": {'ON': 'BPU','InputType': 'int16','OutputType': 'int16'}}
  # YOLO11 l, x
  # node_info: {"/model.10/m/m.0/attn/Softmax": {'ON': 'BPU','InputType': 'int16','OutputType': 'int16'},
  #             "/model.10/m/m.1/attn/Softmax": {'ON': 'BPU','InputType': 'int16','OutputType': 'int16'}}
input_parameters:
  # input_batch: 8
  input_name: ""
  input_type_rt: 'nv12'
  input_type_train: 'rgb'
  input_layout_train: 'NCHW'
  norm_type: 'data_scale'
  scale_value: 0.003921568627451
calibration_parameters:
  cal_data_dir: './calibration_images_640' # 校准数据集文件夹
  cal_data_type: 'float32'
  # preprocess_on: True
  # calibration_type: 'default'
compiler_parameters:
  compile_mode: 'latency'
  debug: False
  optimize_level: 'O3'
复制成功

防止混乱给看下此时的文件夹大概结构,肯定和我的文件数量不太一样,看大概的文件创建位置即可:

文件位置

然后进入docker镜像:

代码块
JavaScript
自动换行
复制代码
# 使用GPU版本镜像
sh run_docker.sh models/ gpu
# 使用CPU版本镜像
sh run_docker.sh models/ cpu
复制成功

8. 验证onnx文件

对转换好的onnx文件进行验证

代码块
JavaScript
自动换行
复制代码
hb_mapper checker --model-type onnx --march bayes-e --model onnx文件的路径.onnx
复制成功

注意看结果一定要为6个输出头:

6个输出头

9. 量化onnx文件

使用的yaml文件是上文(第7步)创建的

代码块
JavaScript
自动换行
复制代码
hb_mapper makertbin --model-type onnx --config config_yolo11_detect_bayese_640x640_nv12.yaml
复制成功

执行后可以看到报了一个错误,但不致命(这里的报错我没有解决,但是不影响最后的使用),有解决方法还望指出

报错

如果你是完全按着我的专栏来操作的话,转换后的bin文件在oe开发包文件夹的bin_dir文件中

bin文件位置

10. 修正bin文件

在docker容器中执行:

代码块
JavaScript
自动换行
复制代码
hb_model_modifier bin模型文件.bin
复制成功

可以看到oe开发包文件夹下多出了一个文件:hb_model_modifier.log

hb_model_modifier.log

打开这个文件找到dim属性的值为64的部分,并且记住name的值,我这里为:output0

记住name的值

因为我的name值为output0,继续往下找,找到下图中output属性为output0的部分,记住这个name的值,我这里的值为:/model.23/cv2.0/cv2.0.2/Conv_output_0_HzDequantize

记住这个name的值

name中包含cv2.x,所以找到这个文件中的最后一行,分别找到cv2.0、cv2.1、cv2.2

分别找到cv2.0、cv2.1、cv2.2

随后执行以下命令:

代码块
JavaScript
自动换行
复制代码
hb_model_modifier bin文件.bin \
-r "/model.23/cv2.0/cv2.0.2/Conv_output_0_HzDequantize" \
-r "/model.23/cv2.1/cv2.1.2/Conv_output_0_HzDequantize" \
-r "/model.23/cv2.2/cv2.2.2/Conv_output_0_HzDequantize"
复制成功

执行后会在你的bin文件的目录中新增一个修改后的bin文件:

新增了一个修改后的bin文件

在docker环境中输入以下命令查看新的bin文件的头信息:

代码块
JavaScript
自动换行
复制代码
hrt_model_exec model_info --model_file 新的bin文件.bin
复制成功

在输出的头信息中有三个scale data属性非常的长,找到并且记住其对应的output[index]中的index的值,我这里的是output[0]、output[1]、output[2],也就是0,1,2

输出的头信息

11. 部署测试

测试单张图片(自行修改相关参数):修改好之后使用原始的bin文件和修改后的bin文件均可

代码块
JavaScript
自动换行
复制代码
#!/user/bin/env python

# Copyright (c) 2024,WuChao D-Robotics.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# 注意: 此程序在RDK板端端运行
# Attention: This program runs on RDK board.

import cv2
import numpy as np
from scipy.special import softmax
# from scipy.special import expit as sigmoid
from hobot_dnn import pyeasy_dnn as dnn  # BSP Python API

from time import time
import argparse
import logging 

# 日志模块配置
# logging configs
logging.basicConfig(
    level = logging.DEBUG,
    format = '[%(name)s] [%(asctime)s.%(msecs)03d] [%(levelname)s] %(message)s',
    datefmt='%H:%M:%S')
logger = logging.getLogger("RDK_YOLO")

def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('--model-path', type=str, default='models/yolo11n_detect_bayese_640x640_nv12.bin', 
                        help="""Path to BPU Quantized *.bin Model.
                                RDK X3(Module): Bernoulli2.
                                RDK Ultra: Bayes.
                                RDK X5(Module): Bayes-e.
                                RDK S100: Nash-e.
                                RDK S100P: Nash-m.""") 
    parser.add_argument('--test-img', type=str, default='../../../resource/assets/bus.jpg', help='Path to Load Test Image.')
    parser.add_argument('--img-save-path', type=str, default='jupyter_result.jpg', help='Path to Load Test Image.')
    parser.add_argument('--classes-num', type=int, default=80, help='Classes Num to Detect.')
    parser.add_argument('--reg', type=int, default=16, help='DFL reg layer.')
    parser.add_argument('--iou-thres', type=float, default=0.45, help='IoU threshold.')
    parser.add_argument('--conf-thres', type=float, default=0.25, help='confidence threshold.')
    opt = parser.parse_args()
    logger.info(opt)

    # 实例化
    model = YOLO11_Detect(opt.model_path, opt.conf_thres, opt.iou_thres)
    # 读图
    img = cv2.imread(opt.test_img)
    # 准备输入数据
    input_tensor = model.bgr2nv12(img)
    # 推理
    outputs = model.c2numpy(model.forward(input_tensor))
    # 后处理
    ids, scores, bboxes = model.postProcess(outputs)
    # 渲染
    logger.info("\033[1;32m" + "Draw Results: " + "\033[0m")
    for class_id, score, bbox in zip(ids, scores, bboxes):
        x1, y1, x2, y2 = bbox
        logger.info("(%d, %d, %d, %d) -> %s: %.2f"%(x1,y1,x2,y2, coco_names[class_id], score))
        draw_detection(img, (x1, y1, x2, y2), score, class_id)
    # 保存结果
    cv2.imwrite(opt.img_save_path, img)
    logger.info("\033[1;32m" + f"saved in path: \"./{opt.img_save_path}\"" + "\033[0m")

class BaseModel:
    def __init__(
        self,
        model_file: str
        ) -> None:
        # 加载BPU的bin模型, 打印相关参数
        # Load the quantized *.bin model and print its parameters
        try:
            begin_time = time()
            self.quantize_model = dnn.load(model_file)
            logger.debug("\033[1;31m" + "Load D-Robotics Quantize model time = %.2f ms"%(1000*(time() - begin_time)) + "\033[0m")
        except Exception as e:
            logger.error("❌ Failed to load model file: %s"%(model_file))
            logger.error("You can download the model file from the following docs: ./models/download.md") 
            logger.error(e)
            exit(1)

        logger.info("\033[1;32m" + "-> input tensors" + "\033[0m")
        for i, quantize_input in enumerate(self.quantize_model[0].inputs):
            logger.info(f"intput[{i}], name={quantize_input.name}, type={quantize_input.properties.dtype}, shape={quantize_input.properties.shape}")

        logger.info("\033[1;32m" + "-> output tensors" + "\033[0m")
        for i, quantize_input in enumerate(self.quantize_model[0].outputs):
            logger.info(f"output[{i}], name={quantize_input.name}, type={quantize_input.properties.dtype}, shape={quantize_input.properties.shape}")

        self.model_input_height, self.model_input_weight = self.quantize_model[0].inputs[0].properties.shape[2:4]

    def resizer(self, img: np.ndarray)->np.ndarray:
        img_h, img_w = img.shape[0:2]
        self.y_scale, self.x_scale = img_h/self.model_input_height, img_w/self.model_input_weight
        return cv2.resize(img, (self.model_input_height, self.model_input_weight), interpolation=cv2.INTER_NEAREST) # 利用resize重新开辟内存
    
    def preprocess(self, img: np.ndarray)->np.array:
        """
        Preprocesses an input image to prepare it for model inference.

        Args:
            img (np.ndarray): The input image in BGR format as a NumPy array.

        Returns:
            np.array: The preprocessed image tensor in NCHW format ready for model input.

        Procedure:
            1. Resizes the image to a specified dimension (`input_image_size`) using nearest neighbor interpolation.
            2. Converts the image color space from BGR to RGB.
            3. Transposes the dimensions of the image tensor to channel-first order (CHW).
            4. Adds a batch dimension, thus conforming to the NCHW format expected by many models.
            Note: Normalization to [0, 1] is assumed to be handled elsewhere based on configuration.
        """
        begin_time = time()

        input_tensor = self.resizer(img)
        input_tensor = cv2.cvtColor(input_tensor, cv2.COLOR_BGR2RGB)
        # input_tensor = np.array(input_tensor) / 255.0  # yaml文件中已经配置前处理
        input_tensor = np.transpose(input_tensor, (2, 0, 1))
        input_tensor = np.expand_dims(input_tensor, axis=0).astype(np.uint8)  # NCHW

        logger.debug("\033[1;31m" + f"pre process time = {1000*(time() - begin_time):.2f} ms" + "\033[0m")
        return input_tensor

    def bgr2nv12(self, bgr_img: np.ndarray) -> np.ndarray:
        """
        Convert a BGR image to the NV12 format.

        NV12 is a common video encoding format where the Y component (luminance) is full resolution,
        and the UV components (chrominance) are half-resolution and interleaved. This function first
        converts the BGR image to YUV 4:2:0 planar format, then rearranges the UV components to fit
        the NV12 format.

        Parameters:
        bgr_img (np.ndarray): The input BGR image array.

        Returns:
        np.ndarray: The converted NV12 format image array.
        """
        begin_time = time()
        bgr_img = self.resizer(bgr_img)
        height, width = bgr_img.shape[0], bgr_img.shape[1]
        area = height * width
        yuv420p = cv2.cvtColor(bgr_img, cv2.COLOR_BGR2YUV_I420).reshape((area * 3 // 2,))
        y = yuv420p[:area]
        uv_planar = yuv420p[area:].reshape((2, area // 4))
        uv_packed = uv_planar.transpose((1, 0)).reshape((area // 2,))
        nv12 = np.zeros_like(yuv420p)
        nv12[:height * width] = y
        nv12[height * width:] = uv_packed

        logger.debug("\033[1;31m" + f"bgr8 to nv12 time = {1000*(time() - begin_time):.2f} ms" + "\033[0m")
        return nv12


    def forward(self, input_tensor: np.array) -> list[dnn.pyDNNTensor]:
        begin_time = time()
        quantize_outputs = self.quantize_model[0].forward(input_tensor)
        logger.debug("\033[1;31m" + f"forward time = {1000*(time() - begin_time):.2f} ms" + "\033[0m")
        return quantize_outputs


    def c2numpy(self, outputs) -> list[np.array]:
        begin_time = time()
        outputs = [dnnTensor.buffer for dnnTensor in outputs]
        logger.debug("\033[1;31m" + f"c to numpy time = {1000*(time() - begin_time):.2f} ms" + "\033[0m")
        return outputs

class YOLO11_Detect(BaseModel):
    def __init__(self, 
                model_file: str, 
                conf: float, 
                iou: float
                ):
        super().__init__(model_file)
        # 将反量化系数准备好, 只需要准备一次
        # prepare the quantize scale, just need to generate once
        self.s_bboxes_scale = self.quantize_model[0].outputs[0].properties.scale_data[np.newaxis, :]
        self.m_bboxes_scale = self.quantize_model[0].outputs[1].properties.scale_data[np.newaxis, :]
        self.l_bboxes_scale = self.quantize_model[0].outputs[2].properties.scale_data[np.newaxis, :]
        logger.info(f"{self.s_bboxes_scale.shape=}, {self.m_bboxes_scale.shape=}, {self.l_bboxes_scale.shape=}")

        # DFL求期望的系数, 只需要生成一次
        # DFL calculates the expected coefficients, which only needs to be generated once.
        self.weights_static = np.array([i for i in range(16)]).astype(np.float32)[np.newaxis, np.newaxis, :]
        logger.info(f"{self.weights_static.shape = }")

        # anchors, 只需要生成一次
        self.s_anchor = np.stack([np.tile(np.linspace(0.5, 79.5, 80), reps=80), 
                            np.repeat(np.arange(0.5, 80.5, 1), 80)], axis=0).transpose(1,0)
        self.m_anchor = np.stack([np.tile(np.linspace(0.5, 39.5, 40), reps=40), 
                            np.repeat(np.arange(0.5, 40.5, 1), 40)], axis=0).transpose(1,0)
        self.l_anchor = np.stack([np.tile(np.linspace(0.5, 19.5, 20), reps=20), 
                            np.repeat(np.arange(0.5, 20.5, 1), 20)], axis=0).transpose(1,0)
        logger.info(f"{self.s_anchor.shape = }, {self.m_anchor.shape = }, {self.l_anchor.shape = }")

        # 输入图像大小, 一些阈值, 提前计算好
        self.input_image_size = 640
        self.conf = conf
        self.iou = iou
        self.conf_inverse = -np.log(1/conf - 1)
        logger.info("iou threshol = %.2f, conf threshol = %.2f"%(iou, conf))
        logger.info("sigmoid_inverse threshol = %.2f"%self.conf_inverse)
    

    def postProcess(self, outputs: list[np.ndarray]) -> tuple[list]:
        begin_time = time()
        # reshape
        s_bboxes = outputs[0].reshape(-1, 64)
        m_bboxes = outputs[1].reshape(-1, 64)
        l_bboxes = outputs[2].reshape(-1, 64)
        s_clses = outputs[3].reshape(-1, 80)
        m_clses = outputs[4].reshape(-1, 80)
        l_clses = outputs[5].reshape(-1, 80)

        # classify: 利用numpy向量化操作完成阈值筛选(优化版 2.0)
        s_max_scores = np.max(s_clses, axis=1)
        s_valid_indices = np.flatnonzero(s_max_scores >= self.conf_inverse)  # 得到大于阈值分数的索引,此时为小数字
        s_ids = np.argmax(s_clses[s_valid_indices, : ], axis=1)
        s_scores = s_max_scores[s_valid_indices]

        m_max_scores = np.max(m_clses, axis=1)
        m_valid_indices = np.flatnonzero(m_max_scores >= self.conf_inverse)  # 得到大于阈值分数的索引,此时为小数字
        m_ids = np.argmax(m_clses[m_valid_indices, : ], axis=1)
        m_scores = m_max_scores[m_valid_indices]

        l_max_scores = np.max(l_clses, axis=1)
        l_valid_indices = np.flatnonzero(l_max_scores >= self.conf_inverse)  # 得到大于阈值分数的索引,此时为小数字
        l_ids = np.argmax(l_clses[l_valid_indices, : ], axis=1)
        l_scores = l_max_scores[l_valid_indices]

        # 3个Classify分类分支:Sigmoid计算
        s_scores = 1 / (1 + np.exp(-s_scores))
        m_scores = 1 / (1 + np.exp(-m_scores))
        l_scores = 1 / (1 + np.exp(-l_scores))

        # 3个Bounding Box分支:筛选
        s_bboxes_float32 = s_bboxes[s_valid_indices,:]#.astype(np.float32) * self.s_bboxes_scale
        m_bboxes_float32 = m_bboxes[m_valid_indices,:]#.astype(np.float32) * self.m_bboxes_scale
        l_bboxes_float32 = l_bboxes[l_valid_indices,:]#.astype(np.float32) * self.l_bboxes_scale

        # 3个Bounding Box分支:dist2bbox (ltrb2xyxy)
        s_ltrb_indices = np.sum(softmax(s_bboxes_float32.reshape(-1, 4, 16), axis=2) * self.weights_static, axis=2)
        s_anchor_indices = self.s_anchor[s_valid_indices, :]
        s_x1y1 = s_anchor_indices - s_ltrb_indices[:, 0:2]
        s_x2y2 = s_anchor_indices + s_ltrb_indices[:, 2:4]
        s_dbboxes = np.hstack([s_x1y1, s_x2y2])*8

        m_ltrb_indices = np.sum(softmax(m_bboxes_float32.reshape(-1, 4, 16), axis=2) * self.weights_static, axis=2)
        m_anchor_indices = self.m_anchor[m_valid_indices, :]
        m_x1y1 = m_anchor_indices - m_ltrb_indices[:, 0:2]
        m_x2y2 = m_anchor_indices + m_ltrb_indices[:, 2:4]
        m_dbboxes = np.hstack([m_x1y1, m_x2y2])*16

        l_ltrb_indices = np.sum(softmax(l_bboxes_float32.reshape(-1, 4, 16), axis=2) * self.weights_static, axis=2)
        l_anchor_indices = self.l_anchor[l_valid_indices,:]
        l_x1y1 = l_anchor_indices - l_ltrb_indices[:, 0:2]
        l_x2y2 = l_anchor_indices + l_ltrb_indices[:, 2:4]
        l_dbboxes = np.hstack([l_x1y1, l_x2y2])*32

        # 大中小特征层阈值筛选结果拼接
        dbboxes = np.concatenate((s_dbboxes, m_dbboxes, l_dbboxes), axis=0)
        scores = np.concatenate((s_scores, m_scores, l_scores), axis=0)
        ids = np.concatenate((s_ids, m_ids, l_ids), axis=0)

        # nms
        indices = cv2.dnn.NMSBoxes(dbboxes, scores, self.conf, self.iou)

        # 还原到原始的img尺度
        bboxes = dbboxes[indices] * np.array([self.x_scale, self.y_scale, self.x_scale, self.y_scale])
        bboxes = bboxes.astype(np.int32)

        logger.debug("\033[1;31m" + f"Post Process time = {1000*(time() - begin_time):.2f} ms" + "\033[0m")

        return ids[indices], scores[indices], bboxes


coco_names = [
    "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", 
    "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", 
    "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", 
    "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard", "tennis racket", "bottle", 
    "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple", "sandwich", "orange", 
    "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", 
    "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", 
    "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush"
    ]

rdk_colors = [
    (56, 56, 255), (151, 157, 255), (31, 112, 255), (29, 178, 255),(49, 210, 207), (10, 249, 72), (23, 204, 146), (134, 219, 61),
    (52, 147, 26), (187, 212, 0), (168, 153, 44), (255, 194, 0),(147, 69, 52), (255, 115, 100), (236, 24, 0), (255, 56, 132),
    (133, 0, 82), (255, 56, 203), (200, 149, 255), (199, 55, 255)]

def draw_detection(img: np.array, 
                   bbox: tuple[int, int, int, int],
                   score: float, 
                   class_id: int) -> None:
    """
    Draws a detection bounding box and label on the image.

    Parameters:
        img (np.array): The input image.
        bbox (tuple[int, int, int, int]): A tuple containing the bounding box coordinates (x1, y1, x2, y2).
        score (float): The detection score of the object.
        class_id (int): The class ID of the detected object.
    """
    x1, y1, x2, y2 = bbox
    color = rdk_colors[class_id%20]
    cv2.rectangle(img, (x1, y1), (x2, y2), color, 2)
    label = f"{coco_names[class_id]}: {score:.2f}"
    (label_width, label_height), _ = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.5, 1)
    label_x, label_y = x1, y1 - 10 if y1 - 10 > label_height else y1 + 10
    cv2.rectangle(
        img, (label_x, label_y - label_height), (label_x + label_width, label_y + label_height), color, cv2.FILLED
    )
    cv2.putText(img, label, (label_x, label_y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0), 1, cv2.LINE_AA)

if __name__ == "__main__":
    main()
复制成功

修改哪里呢?

大概是第49行

代码块
JavaScript
自动换行
复制代码
parser.add_argument('--classes-num', type=int, default=你的模型检测的数量, help='Classes Num to Detect.')
复制成功

大概是第290行

代码块
JavaScript
自动换行
复制代码
coco_names = [改成你的检测信息]
复制成功

大概是第221行

代码块
JavaScript
自动换行
复制代码
s_bboxes = outputs[0].reshape(-1, 64)
m_bboxes = outputs[1].reshape(-1, 64)
l_bboxes = outputs[2].reshape(-1, 64)
s_clses = outputs[3].reshape(-1, 你的识别项数量)
m_clses = outputs[4].reshape(-1, 你的识别项数量)
l_clses = outputs[5].reshape(-1, 你的识别项数量)

# 如果你的不是0,1,2,假如是0,2,4
# s_bboxes = outputs[0].reshape(-1, 64)
# m_bboxes = outputs[2].reshape(-1, 64)
# l_bboxes = outputs[4].reshape(-1, 64)
# s_clses = outputs[1].reshape(-1, 你的识别项数量)
# m_clses = outputs[3].reshape(-1, 你的识别项数量)
# l_clses = outputs[5].reshape(-1, 你的识别项数量)
复制成功

如果有遗漏的地方需要补充的还请指出

12. 本专栏参考文章:

https://forum.d-robotics.cc/t/topic/27155

https://forum.d-robotics.cc/t/topic/28039

https://forum.d-robotics.cc/t/topic/28035

https://forum.d-robotics.cc/t/topic/28624

https://forum.d-robotics.cc/t/topic/28641

https://forum.d-robotics.cc/t/topic/28935

https://horizonrobotics.feishu.cn/docx/QYfCddrK8o2IIJxVUypcv3b7nsf

https://horizonrobotics.feishu.cn/docx/Oo5WdSyApoJyfOx2psicZ7kpndd