Run the Image Classification Sample Application - Qualcomm Dragonwing Documentation

The gst-ai-classification sample application demonstrates hardware-accelerated image classification on a video stream. The pipeline receives input from a camera, file source, Real-Time Streaming Protocol (RTSP) stream or USB camera, performs preprocessing, runs inference on the AI hardware, and displays the results on screen.

The gst-ai-classification application is part of the Qualcomm Intelligent Multimedia (QIM) SDK and is available on the device after flashing. You must push the model and label files to the device before running the application.

Download model and label files

Enable Wi-Fi and SSH on the device

The device requires an internet connection to download the artifacts needed to run sample applications. If SSH and Wi-Fi are already configured, skip this step.Follow Setup an SSH connection to enable Wi-Fi and SSH on the device.

Set the user environment variable on the host computer

export USER=root

ssh $USER@<IP_ADDRESS_OF_TARGET_DEVICE>

Download and run the download_artifacts_2.x.sh script

cd /tmp
curl -L -O https://raw.githubusercontent.com/qualcomm/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/download_artifacts_2.x.sh
chmod +x download_artifacts_2.x.sh 
./download_artifacts_2.x.sh

Enable qticamsrc

In the terminal of the target device, run the following command to enable camera:

echo -n "camx" > /var/data
efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -w -f /var/data
efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -p
sync
reboot

camera use-case is not supported on IQ-2390.

(Optional) Download the YOLOv8 model

Use one of the following methods:

Download with a script
Export with AI Hub APIs
Generate a batch model

Create a Qualcomm AI Hub account

Create a Qualcomm AI Hub account.

Copy your API key

Go to Settings in the upper right corner and copy your API key.

Run the export script on the host computer

curl -L -O https://raw.githubusercontent.com/qualcomm/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/export_model.sh
chmod +x export_model.sh

Replace <API_KEY> with your API key:

./export_model.sh --api-key=<API_KEY>

The models are downloaded to the export_assets directory.

Copy the model to the ${HOME}/Downloads/qimsdk_samples/models/ directory on the device

scp <working-directory>/export_assets/yolov8_det-tflite-w8a8/yolov8_det.tflite $USER@<IP_ADDRESS_OF_TARGET_DEVICE>:~/Downloads/qimsdk_samples/models/

Use the YOLOv8-Detection-Quantized model from AI Hub.Current releases use the following SDK versions:

Qualcomm Linux (GA 2.1): Qualcomm AI Runtime SDK v2.50.0.260828

For example, to export the YOLOv8 QNN model, run:

qai-hub-models export yolov8_det --quantize w8a8 --target-runtime=qnn_context_binary --device="Dragonwing RB3 Gen 2 Vision Kit" --compile-options="--qairt_version 2.50" --profile-options "--qairt_version 2.50"

Update --qairt_version to match the version installed on your target device. To check the installed version:

qairt-net-run --version

To list the QAIRT SDK versions supported by AI Hub for the export command, run this command on the host device:

qai-hub list-frameworks

To list supported devices, run this command on the host device:

qai-hub list-devices

For example, to export the YOLOv8 LiteRT model, run:

qai-hub-models export yolov8_det --quantize w8a8 --target-runtime=tflite --device="Dragonwing RB3 Gen 2 Vision Kit"

To change the batch size, update <N> in the following export command:

qai-hub-models export <Model_Name> --batch-size <N> --device="Dragonwing RB3 Gen 2 Vision Kit"

For example, to export the YOLOv8 LiteRT model with batch size 4:

qai-hub-models export yolov8_det --quantize w8a8 --target-runtime=tflite --device="Dragonwing RB3 Gen 2 Vision Kit" --batch-size 4

Run the sample application

On the host computer, set the user environment variable

export USER=root

ssh $USER@<IP_ADDRESS_OF_TARGET_DEVICE>

Run the application with the sample configuration file

To run the application with the sample configuration file, use:

gst-ai-classification --config-file=/etc/configs/config_classification.json

This command runs the application using the sample configuration in /etc/configs/config_classification.json.To view the available command-line options, run:

gst-ai-classification -h

Press CTRL + C to stop the execution.

Run the application with a custom configuration file (optional)

You can use a custom configuration file by specifying its path with --config-file:

gst-ai-classification --config-file=/path/to/config_classification.json

If --config-file is specified, the application uses that file. Otherwise, it searches for config_classification.json in the following order:

Priority	Location	Description
1	The path provided with `--config-file`.	Uses the configuration file specified on the command line.
2	`$XDG_CONFIG_HOME/qimsdk_samples/config_classification.json`	Uses the user-defined configuration directory when the environment variable is set.
3	`$HOME/.config/qimsdk_samples/config_classification.json`	Uses the default user configuration directory when no file is found in `XDG_CONFIG_HOME`.
4	`/etc/configs/config_classification.json`	Uses the system-wide configuration file as the final fallback.

To create a user-specific configuration, copy the sample file to the user configuration directory:

export XDG_CONFIG_HOME="$HOME/.config"
mkdir -p "$XDG_CONFIG_HOME/qimsdk_samples"
cp /etc/configs/config_classification.json "$XDG_CONFIG_HOME/qimsdk_samples/config_classification.json"

Modify the copied configuration file to specify the required parameters. The custom configuration file must follow the schema defined in configurations.

Run the application:

gst-ai-classification

The application overlays classified object labels and confidence scores on the video stream and displays the results on the configured output. To stop the application, press Ctrl+C.

Configure the application

Edit the config_classification.json file to specify your model, input source, and output preferences. Configuration template

{
  "file-path": "<path-to-input-video>",
  "ml-framework": "<snpe or tflite or qnn or onnx>",
  "model": "<path-to-model-file>",
  "labels": "<path-to-label-file>",
  "threshold": "<postprocessing threshold, integer value from 1 to 100>",
  "runtime": "<dsp, gpu, or cpu>",
  "output-type": "<waylandsink, filesink, or rtspsink>"
}

Example configuration — LiteRT model on a video file using the DSP runtime:

{
  "file-path": "video.mp4",
  "ml-framework": "tflite",
  "model":" inception_v3_quantized.tflite",
  "labels": "classification.json",
  "threshold": 40,
  "runtime": "dsp",
  "output-type": "waylandsink"
}

This section uses the following default file locations:

${HOME}/Downloads/qimsdk_samples/models/ — model files
${HOME}/Downloads/qimsdk_samples/labels/ — label files
${HOME}/Downloads/qimsdk_samples/media/ — video files

JSON configuration field descriptions

Field	Description
`ml-framework`	`snpe`: Qualcomm Neural Processing SDK `tflite`: LiteRT `qnn`: Qualcomm AI Engine Direct `onnx`: ONNX Runtime
`runtime`	`cpu` `gpu` `dsp`
`output-ip-address`	Output server IP address
`port`	Output server port
`video-format`	USB camera format: `nv12`, `yuy2`, or `mjpeg`, with `width`, `height`, and `framerate` parameters
`output-file`	Name of the output file. Default: `output_classification.mp4`
`output-type`	`waylandsink`: display on Wayland `filesink`: save to file `rtspsink`: stream to RTSP server
`file-path`	Path to the input video file
`rtsp-ip-port`	RTSP input stream in the format `rtsp://<ip>:<port>/<stream>`
`camera`	Camera index: primary (`0`) or secondary (`1`)

Notes

To stop the application, press Ctrl+C.
To enable GStreamer debug logging, set the GST_DEBUG environment variable. For example, to log all warnings:
export GST_DEBUG=2
For more troubleshooting options, see Troubleshooting.

​Download model and label files

​Run the sample application

​Configure the application

​Notes

Download model and label files

Run the sample application

Configure the application

Notes