Run a Sample Application with a custom-trained model - Qualcomm Dragonwing Documentation

This guide explains how to modify an existing Qualcomm IM SDK reference application to work with a custom-trained model. It uses a custom-trained YOLOv8 model as an example.

Use a custom-trained YOLOv8 LiteRT model

Qualcomm IM SDK reference applications use the YOLOv8 model for object detection. This example explains how to run a custom-trained YOLOv8 variant with the current reference application. To run your own custom-trained YOLOv8 model, complete the following steps:

Replace the existing model with your new model in the reference application.
Update the label files with your custom labels.
Run the reference application with the updated model.

Update the label files

The Qualcomm IM SDK reference applications expect labels in JSON format. Update the id, color, and label values for each entry in the labels file. The format for each label entry is:

[
  {"id": 0, "color": "0x00FF00FF", "label": "person"}
]

For example:

[
  {"id": 0, "color": "0x00FF00FF", "label": "person"},
  {"id": 1, "color": "0x00FF00FF", "label": "bicycle"},
  {"id": 2, "color": "0x0000FFFF", "label": "car"},
  {"id": 3, "color": "0x00FF00FF", "label": "motorcycle"}
]

Run object detection with the custom model

To run object detection using the LiteRT runtime with your custom model and label files, complete the following steps:

Set the user environment variable on the host computer

export USER=root

Create the artifact directories

ssh $USER@<IP_ADDRESS_OF_TARGET_DEVICE>

mkdir -p ${HOME}/Downloads/qimsdk_samples/media ${HOME}/Downloads/qimsdk_samples/labels ${HOME}/Downloads/qimsdk_samples/models

Copy the model to the device

scp yolov8_custom.tflite $USER@<IP_ADDRESS_OF_TARGET_DEVICE>:~/Downloads/qimsdk_samples/models/

Copy the label file to the device

scp yolov8_custom.json $USER@<IP_ADDRESS_OF_TARGET_DEVICE>:~/Downloads/qimsdk_samples/labels/

ssh $USER@<IP_ADDRESS_OF_TARGET_DEVICE>

Run the object detection application with sample configuration file

gst-ai-object-detection --config-file=/etc/configs/config_detection.json

This command runs the application using the sample configuration in /etc/configs/config_detection.json.To stop the application, press Ctrl+C.

(Optional) Run the object detection application with a custom configuration file

You can use a custom configuration file by specifying its path with --config-file:

gst-ai-object-detection --config-file=/path/to/config_detection.json

If --config-file is specified, the application uses that file. Otherwise, it searches for config_detection.json in the following order:

Priority	Location	Description
1	The path provided with `--config-file`.	Uses the configuration file specified on the command line.
2	`$XDG_CONFIG_HOME/qimsdk_samples/config_detection.json`	Uses the user-defined configuration directory when the environment variable is set.
3	`$HOME/.config/qimsdk_samples/config_detection.json`	Uses the default user configuration directory when no file is found in `XDG_CONFIG_HOME`.
4	`/etc/configs/config_detection.json`	Uses the system-wide configuration file as the final fallback.

To create a user-specific configuration, copy the sample file to the user configuration directory:

export XDG_CONFIG_HOME="$HOME/.config"
mkdir -p "$XDG_CONFIG_HOME/qimsdk_samples"
cp /etc/configs/config_detection.json "$XDG_CONFIG_HOME/qimsdk_samples/config_detection.json"

To view the available command-line options, run:

gst-ai-object-detection -h

Edit the config_detection.json configuration file

{
   "file-path": "video.mp4",
   "ml-framework": "tflite",
   "yolo-model-type": "yolox",
   "model": "yolox_quantized.tflite",
   "labels": "yolox.json",
   "threshold": 40,
   "runtime": "dsp",
   "output-type": "waylandsink"
}

The application uses $HOME/Downloads/qimsdk_samples as its artifact directory. When a relative path or filename is specified, it is resolved against the corresponding subdirectory below. Alternatively, absolute paths are used as provided.

Configuration field	Resolution of a relative path
`file-path`	`$HOME/Downloads/qimsdk_samples/media/video.mp4`
`model`	`$HOME/Downloads/qimsdk_samples/models/yolov8_custom.tflite`
`labels`	`$HOME/Downloads/qimsdk_samples/labels/yolov8_custom.json`

Run the application:

gst-ai-object-detection

When running, the application displays the video stream on the connected monitor with bounding box results overlaid on each frame.

​Use a custom-trained YOLOv8 LiteRT model

​Update the label files

​Run object detection with the custom model

Use a custom-trained YOLOv8 LiteRT model

Update the label files

Run object detection with the custom model