> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs-staging.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

This guide covers AI application development using the tools, runtimes, and frameworks supported on Qualcomm Dragonwing IoT platforms. It is intended for developers who want to train or fine-tune models, prepare them for deployment, and build AI applications on Qualcomm hardware.

Most decisions in this guide come down to three questions:

* **What are you building?** Classic ML/vision, generative AI (LLM/VLM), or voice/speech AI — each has its own tools and quick-start paths.
* **How will you prepare your model?** Convert, quantize, compile, or fine-tune it for the target hardware.
* **How will you run inference?** The runtime, container, or integration path that executes the model on the device.

You can bring pretrained models from ONNX, PyTorch, TensorFlow, or LiteRT — or start from Qualcomm's ready-to-run LLM, VLM, ASR, and TTS models — and run them efficiently across the Qualcomm Kryo™ CPU, Adreno™ GPU, and Hexagon™ NPU (HTP).

<Note>
  This document assume that the device setup has already been completed. If not, please follow the instructions in the Device Setup and Software Update section of EVK before proceeding.
</Note>

## Choose your journey

Use the flowchart below to find the path that fits your application. It walks you through whether you already have a model, what you're building, how to prepare it, and how to run inference, then links you to the right Ubuntu workflow page. Every highlighted box is clickable.

```mermaid theme={null}
%%{init: {"flowchart": {"rankSpacing": 110, "nodeSpacing": 65}} }%%
flowchart TD
    Q1{"Do you already have<br/>a trained model?"}

    Q1 -->|"No, just show me<br/>what it can do"| RunPrebuilt["Run prebuilt AI models & apps<br/>Qdemo UI · prebuilt samples · IM SDK"]
    Q1 -->|"I want to fine tune a model<br/>with my own data"| EI["Edge Impulse"]
    Q1 -->|"Yes / bring my own<br/>ONNX · PyTorch · TF · LiteRT"| TypeQ{"What are you building?"}

    TypeQ -->|"Classic ML / vision"| Prep{"Prepare the model"}
    TypeQ -->|"Generative AI<br/>LLM · VLM"| GenAI["On-device GenAI workflows<br/>Genie · Llama.cpp · OpenAI-compatible container"]
    TypeQ -->|"Voice / speech AI"| VoiceQ{"What do you need?"}

    VoiceQ -->|"ASR + TTS +<br/>translation"| VoiceContainer["Audio Analytics container"]
    VoiceQ -->|"Specifically<br/>Whisper ASR"| Whisper["Whisper<br/>VoiceAI ASR (NPU) or whisper.cpp (CPU/GPU)"]

    Prep -->|"Ready model, or BYOM<br/>in the cloud"| AIHub["Qualcomm AI Hub<br/>ready models · BYOM<br/>(optimize in the cloud)"]
    Prep -->|"Convert + quantize<br/>my own (local)"| QAIRT["Qualcomm AI Runtime SDK<br/>(QAIRT)"]
    Prep -->|"Recover quantized<br/>accuracy"| AIMET["AIMET (PTQ / QAT)"]

    AIHub --> Runtime{"Run inference"}
    QAIRT --> Runtime
    AIMET --> Runtime

    Runtime -->|"Python / quick .tflite"| LiteRT["LiteRT / TFLite<br/>AI Engine Direct delegate"]
    Runtime -->|"ONNX models"| ONNXRun["ONNX Runtime<br/>AI Engine Direct"]
    Runtime -->|"C++ / low-level control<br/>SNPE or QNN"| QAIRTcpp["QAIRT SDK C++ APIs"]
    Runtime -->|"Camera / video +<br/>AI pipeline"| IMSDK["Qualcomm IM SDK<br/>GStreamer · zero-copy · GPU pre/post"]

    click RunPrebuilt "../map/run-prebuilt-models-and-apps"
    click EI "../topic/edge-impulse"
    click GenAI "../map/run-on-device-genai"
    click VoiceContainer "../map/run-on-device-voice-ai"
    click Whisper "../topic/whisper"
    click AIHub "../topic/ai-hub"
    click QAIRT "../topic/qairt"
    click AIMET "../topic/aimet"
    click LiteRT "../topic/litert-overview"
    click ONNXRun "../topic/onnxruntime"
    click QAIRTcpp "../topic/develop-your-own-application-qairt-cpp"
    click IMSDK "../topic/develop-your-own-application-im-sdk"

    classDef prep fill:#31017D,stroke:#31017D,color:#fff;
    classDef run fill:#3253DC,stroke:#3253DC,color:#fff;
    classDef genai fill:#B45309,stroke:#B45309,color:#fff;
    classDef voice fill:#0F766E,stroke:#0F766E,color:#fff;
    class AIHub,QAIRT,AIMET,EI prep;
    class RunPrebuilt,LiteRT,ONNXRun,QAIRTcpp,IMSDK run;
    class GenAI genai;
    class VoiceContainer,Whisper voice;
```

<Note>
  On-device generative AI availability depends on your Dragonwing development board and model. Start with [GenieX](../topic/geniex), an on device Gen AI inference runtime which either uses llama.cpp runtime (GGML kernels for CPU / GPU / Hexagon HTP) or the Qualcomm® AI Engine Direct runtime (NPU-only). For voice/speech AI, see the [Voice AI workflow](../map/run-on-device-voice-ai) and [Whisper](../topic/whisper).
</Note>

## Application development & execution flow summary

| Flow | Purpose |
| - | - |
| [Edge Impulse](../topic/edge-impulse) | Build and train AI models using audio, image, and other sensor data, or bring your own model in a variety of formats. |
| [Qualcomm IM SDK](../topic/develop-your-own-application-im-sdk) | Build high-performance camera and video AI pipelines using GStreamer, zero-copy buffers, and GPU pre/post-processing. |
| [GenieX](../topic/geniex) | Run frontier language and vision-language models locally on Hexagon NPU, Adreno GPU, or CPU with a few lines of code. |
| [Qualcomm AI Hub](../topic/ai-hub) | Download preoptimized models, or bring your own model (BYOM) and optimize, validate, and deploy on hosted Qualcomm platform devices within minutes. |
| [Qualcomm AI Runtime SDK (QAIRT)](../topic/qairt) | Convert, quantize, and compile models from TensorFlow, PyTorch, LiteRT, or ONNX locally for deployment on the NPU. |
| [AIMET (PTQ / QAT)](../topic/aimet) | Recover accuracy lost during quantization using post-training quantization or quantization-aware training. |
| [LiteRT / TFLite](../topic/litert-overview) | Run `.tflite` models on the NPU (Python or C++) using AI Engine Direct delegates. Works with models from AI Hub, QAIRT, or Edge Impulse. |
| [ONNX Runtime](../topic/onnxruntime) | Run ONNX models on the NPU using ONNX Runtime with AI Engine Direct. |
| [Run models with SNPE or QNN](../topic/run-models) | Run converted `.dlc` or `.so` files on CPU, GPU, or HTP using the QAIRT SDK C/C++ APIs. |
| [Llama.cpp](../topic/llama-cpp) | Run LLMs and VLMs locally using a C++ backend optimized for CPU/GPU and quantized formats. |
| [LLM/VLM using Qualcomm OpenAI container](../topic/llm-vlm-qcom-setup) | Run LLMs/VLMs behind a drop-in OpenAI-compatible API on the NPU. |
| [Whisper](../topic/whisper) | Speech transcription, translation, and language identification using the NPU (VoiceAI ASR) or CPU (whisper.cpp). |
| [Audio Analytics container](../topic/audio-analytics-overview) | Speech recognition (ASR), text-to-speech (TTS), and language translation on the NPU. |

## Prepare your model

Before a model runs efficiently on Qualcomm hardware, it is converted to an executable format and, for the Hexagon NPU (HTP), quantized to a supported precision. (LiteRT models are an exception: they run directly through the AI Engine Direct delegate.) Choose the preparation tool that matches your starting point.

| Tool | Use it to | Output |
| - | - | - |
| [Qualcomm AI Hub](../topic/ai-hub) | Download a preoptimized model, or **bring your own model (BYOM)** and have QAIRT **compile, convert, and quantize it in the cloud** for your target chipset, with no local toolchain required. | Ready-to-run LiteRT or Qualcomm AI Engine Direct model |
| [Qualcomm AI Runtime SDK (QAIRT)](../topic/qairt) | The **local** alternative to AI Hub BYOM: convert, quantize, and compile models from TensorFlow, PyTorch, LiteRT, or ONNX yourself. Integrates AI Engine Direct and the Neural Processing SDK. | Quantized model / compiled context binary |
| [Qualcomm AI Model Efficiency Toolkit](../topic/aimet) | Recover accuracy lost during quantization using post-training quantization (PTQ) and quantization-aware training (QAT). | Higher-accuracy quantized model |
| [Edge Impulse](../topic/edge-impulse) | Build, train, or fine-tune models from your own audio, image, and sensor data. | Trained model in your chosen format |

## Run inference

After your model is prepared, choose how to execute it on the device. The runtime you pick depends on your language, model format, and whether you are building a full camera/video pipeline.

| Runtime / integration | Best for |
| - | - |
| [LiteRT](../topic/litert-overview) | High-performance on-device inference from Python or C++, using Qualcomm AI Engine Direct delegates. |
| [QAIRT SDK C++ APIs](../topic/develop-your-own-application-qairt-cpp) | Low-level C++ control over model execution and the inference backend. |
| [Qualcomm IM SDK](../topic/develop-your-own-application-im-sdk) | High-performance camera, video, and vision pipelines that combine capture, preprocessing, inference, and rendering, with zero-copy buffers and GPU pre/post-processing. |
| [Qualcomm® GenAI Inference Engine (Genie)](https://docs.qualcomm.com/doc/80-63442-10/topic/index_Genie.html) | Running and orchestrating on-device generative AI (LLMs, multimodal) workflows. |
| [Geniex](../topic/geniex) | Fastest way to run LLM/VLM text, image, and audio inference on the NPU via the GenieX CLI. |
| [Llama.cpp](../topic/llama-cpp) | Run LLMs and VLMs locally using a C++ backend optimized for CPU/GPU and quantized formats. |

<Note>
  Building a robotics application? The [Qualcomm® Intelligent Robotics (QIR) SDK](https://www.thundercomm.com/rubik-pi-3/en/docs/rubik-pi-3-user-manual/1.0.0-u/Application%20Development%20and%20Execution%20Guide/Robotics-Sample-Applications/Robotics%20Sample%20Applications/) adds ROS-based modules and hardware-accelerated nodes on top of these runtimes.
</Note>

## AI hardware

Qualcomm platforms include the following hardware accelerators for AI inference:

* **Qualcomm Kryo™ CPU** — High-performance CPU with best-in-class power efficiency.
* **Qualcomm Adreno™ GPU**: Balanced power and performance for AI workloads, accelerated with OpenCL kernels. Also used for model **pre- and post-processing** (for example, the IM SDK runs resize, color conversion, and overlay on the GPU).
* **Qualcomm Hexagon™ Tensor Processor (HTP)**: Also known as NPU/DSP/HMX. Optimized for low-power, high-performance AI **inference**. For best performance, quantize pretrained models to a supported precision.