Detect hands with sample_hand_detection - Qualcomm Dragonwing Documentation

The sample_hand_detection sample application detects hands and hand landmarks from images or camera streams using Python-based ROS 2 nodes. It uses Qualcomm® AI Engine Direct SDK (QNN) for model inference. The model is sourced from the MediaPipe Hand Landmark Detector, which is a machine learning pipeline that predicts bounding boxes and pose skeletons of hands in an image. This sample application allows you to input either a static image named input_image.jpg or a live camera stream, and then it publishes the result as ROS topic /handlandmark_result.

NoteFor more information, see sample_hand_detection on GitHub.

Figure: Sample effects of sample_hand_detection

Pipeline flow for `sample_hand_detection`

Figure: Pipeline flow for sample_hand_detection

ROS nodes used in `sample_hand_detection`

ROS node	Description
qrb_ros_camera	Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics.
qrb_ros_nn_inference	Loads a trained AI model, receives preprocessed images, performs inference, and publishes results.
`qrb_ros_hand_detector`	Subscribes to an image topic (from static image or camera), performs preprocessing and postprocessing, and publishes detection results.
`palm_preprocessor`	Prepares input image data for palm detection by resizing, normalizing images, and converting image formats to match model requirements.
`palm_postprocessor`	Interprets raw output from palm detection model, extracts bounding boxes and confidence scores, applies filtering and optional NMS, and maps results to original image coordinates.
`landmark_preprocessor`	Processes the cropped hand regions based on palm detection results. Resizes and normalizes the image, and may apply geometric transformations to align the hand orientation.
`landmark_postprocessor`	Decodes the output of the landmark detection model, extracts hand keypoints and maps them back to the original image space.

ROS topics used in `sample_hand_detection`

ROS Topic	Type	Published By
`/handlandmark_result`	`sensor_msgs.msg.Image`	`qrb_ros_hand_detector`
`/palm_detector_input_tensor`	`qrb_ros_tensor_list_msgs.msg.TensorList`	`qrb_ros_hand_detector`
`/palm_detector_output_tensor`	`qrb_ros_tensor_list_msgs.msg.TensorList`	`qrb_ros_nn_inference`
`/landmark_detector_input_tensor`	`qrb_ros_tensor_list_msgs.msg.TensorList`	`qrb_ros_hand_detector`
`/landmark_detector_output_tensor`	`qrb_ros_tensor_list_msgs.msg.TensorList`	`qrb_ros_nn_inference`
`/image_raw`	`sensor_msgs.msg.Image`	`qrb_ros_camera`

Prerequisites

You have completed the settings in Set up the environment for running sample applications.

Run out-of-the-box `sample_hand_detection`

Try me

Run the sample on the development kit

Select the input source of the pipeline, and then complete the steps in the corresponding tab.

Image publisher
qrb_ros_camera

Set up the runtime environment.

SSH Session

# Set up the runtime environment
source /usr/share/qirp-setup.sh -m
# Set the ROS_DOMAIN_ID
export ROS_DOMAIN_ID=123
export PYTHONPATH=$PYTHONPATH:/opt/ros/jazzy/lib/python3.14/site-packages/sample_hand_detection

Launch the sample with image_publisher. You can replace image_path with the path to your desired image.

SSH Session

ros2 launch sample_hand_detection launch_with_image_publisher.py model_path:=/opt/model/ \
    image_path:=/usr/ros/jazzy/share/sample_hand_detection/input_image.jpg

Ensure that you have connected a camera to the development kit and set it up according to Set up the camera. The launch file starts the qrb_ros_camera node with camera ID 0 and a 640 × 480 stream, so you don’t need to run qrb_ros_camera separately.

Set up the runtime environment.

SSH Session

# Set up the runtime environment
source /usr/share/qirp-setup.sh -m
# Set the ROS_DOMAIN_ID
export ROS_DOMAIN_ID=123
export PYTHONPATH=$PYTHONPATH:/opt/ros/jazzy/lib/python3.14/site-packages/sample_hand_detection

Launch the sample with qrb_ros_camera.

SSH Session

ros2 launch sample_hand_detection launch_with_qrb_ros_camera.py model_path:=/opt/model/

View the results on the host computer

Start the rqt.

SSH Session

# YOUR_HOST_IP is the IP address of the Host where you want to view the sample output.
export DISPLAY=YOUR_HOST_IP:0
export ROS_DOMAIN_ID=123
rqt

Select the following buttons in sequence: Plugins —> Visualization —> Image View
Select handlandmark_result to see the results.

Build and run `sample_hand_detection`

The following steps build the sample_hand_detection package using Dragonwing IQ-9075 Evaluation Kit as an example.

Try me

Build and package on the host computer

Build the sample application project.

cd <decompressed_workspace>/images/iq-9075-evk/qirpsdk_artifacts/iq-9075-evk/
tar -zxvf qirp-sdk_<qirp_version>.tar.gz
cd qirp-sdk
source setup.sh
# build sample
cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_hand_detection
colcon build

Package and push the sample application to the device.

# package and push build result of sample
cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_hand_detection/install/sample_hand_detection
tar -czvf sample_hand_detection.tar.gz lib share
scp sample_hand_detection.tar.gz root@[ip-addr]:/opt/

Install and run on the development kit

Install the sample application.

SSH Session

# Install sample package
tar --no-overwrite-dir --no-same-owner -zxf /opt/sample_hand_detection.tar.gz -C /usr/ros/jazzy/

Run the sample application with the steps in Run out-of-the-box sample_hand_detection.

Robotics SDK(Linux)

​Pipeline flow for sample_hand_detection

​ROS nodes used in sample_hand_detection

​ROS topics used in sample_hand_detection

​Prerequisites

​Run out-of-the-box sample_hand_detection

​Build and run sample_hand_detection

Pipeline flow for `sample_hand_detection`

ROS nodes used in `sample_hand_detection`

ROS topics used in `sample_hand_detection`

Prerequisites

Run out-of-the-box `sample_hand_detection`

Build and run `sample_hand_detection`