> This page is for version dev.
> For other versions, use one of these documentation indexes:
> - Latest (v1.6.1) (default): https://docs.nvidia.com/dynamo/latest/llms.txt
> - dev: https://docs.nvidia.com/dynamo/dev/llms.txt
> - v1.6.1: https://docs.nvidia.com/dynamo/v1.6.1/llms.txt
> - v1.6.0: https://docs.nvidia.com/dynamo/v1.6.0/llms.txt
> - v1.5.1: https://docs.nvidia.com/dynamo/v1.5.1/llms.txt
> - v1.5.0: https://docs.nvidia.com/dynamo/v1.5.0/llms.txt
> - v1.4.2: https://docs.nvidia.com/dynamo/v1.4.2/llms.txt
> - v1.4.1: https://docs.nvidia.com/dynamo/v1.4.1/llms.txt
> - v1.4.0: https://docs.nvidia.com/dynamo/v1.4.0/llms.txt
> - v1.3.0: https://docs.nvidia.com/dynamo/v1.3.0/llms.txt
> - v1.2.1: https://docs.nvidia.com/dynamo/v1.2.1/llms.txt
> - v1.2.0: https://docs.nvidia.com/dynamo/v1.2.0/llms.txt
> - v1.1.1: https://docs.nvidia.com/dynamo/v1.1.1/llms.txt
> - v1.1.0: https://docs.nvidia.com/dynamo/v1.1.0/llms.txt
> - v1.0.2: https://docs.nvidia.com/dynamo/v1.0.2/llms.txt
> - v1.0.1: https://docs.nvidia.com/dynamo/v1.0.1/llms.txt
> - v1.0.0: https://docs.nvidia.com/dynamo/v1.0.0/llms.txt
> - v0.9.1: https://docs.nvidia.com/dynamo/v-0-9-1/llms.txt
> - v0.9.0: https://docs.nvidia.com/dynamo/v-0-9-0/llms.txt
> - v0.8.1: https://docs.nvidia.com/dynamo/v-0-8-1/llms.txt
> - v0.8.0: https://docs.nvidia.com/dynamo/v-0-8-0/llms.txt
> - v0.7.1: https://docs.nvidia.com/dynamo/v-0-7-1/llms.txt
> - v0.7.0: https://docs.nvidia.com/dynamo/v-0-7-0/llms.txt

> For clean Markdown content of this page, append .md to this URL. For the complete documentation index, see https://docs.nvidia.com/dynamo/llms.txt. For full content including API reference and SDK examples, see https://docs.nvidia.com/dynamo/llms-full.txt.

# Image Understanding

Dynamo serves vision language models (VLMs) through the OpenAI-compatible
`/v1/chat/completions` endpoint. A request carries one or more `image_url`
content parts next to its text, and the worker runs the model's vision encoder
before prefill.

This page gets one image request working on each backend. To generate images
instead, see [Text to Image](/dynamo/dev/diffusion/text-to-image). To speed up
image workloads, see [VLM Performance Optimization](/dynamo/dev/image-understanding/vlm-performance-optimization).

## Support Matrix

| Input | vLLM | SGLang | TensorRT-LLM |
| --- | --- | --- | --- |
| `https://` image URL | Yes | Yes | Yes |
| `data:` (base64) image | Yes | Yes | Yes |
| Pre-computed embeddings (`.safetensors`) | No | No | Yes |

Plain `http://` URLs and hosts that resolve to private or loopback addresses are
refused by default. See [Media URL Policy](#media-url-policy).

## Set up the backend

The vLLM and TensorRT-LLM launchers below start the worker with
`--enable-multimodal`. The SGLang aggregated launcher (`agg_vision.sh`) does not
pass it, while the SGLang disaggregated launchers do. See
[SGLang Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)
for the flags each worker role needs.

Run the commands from the root of a Dynamo repository checkout. Each launcher
starts a frontend on port 8000 and one aggregated worker on GPU 0.

<Tabs>
  <Tab title="vLLM" language="vllm">
    Requires a working [vLLM backend setup](/dynamo/dev/knowledge-base/modular-components/backends/v-llm/overview).

    ```bash
    bash examples/backends/vllm/launch/agg_multimodal.sh --model Qwen/Qwen3-VL-2B-Instruct
    ```

    The launcher defaults to `Qwen/Qwen3-VL-30B-A3B-Instruct-FP8` when `--model`
    is omitted. For XPU devices, use `launch/xpu/agg_multimodal_xpu.sh`. See
    [vLLM Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/v-llm/vllm-multimodal)
    for disaggregated layouts, LoRA adapters, and supported models.
  </Tab>
  <Tab title="SGLang" language="sglang">
    Requires a working [SGLang backend setup](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/overview).

    ```bash
    bash examples/backends/sglang/launch/agg_vision.sh --model-path Qwen/Qwen3-VL-2B-Instruct
    ```

    See [SGLang Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/sg-lang/sglang-multimodal)
    for disaggregated layouts and supported models.
  </Tab>
  <Tab title="TensorRT-LLM" language="trtllm">
    Requires a working [TensorRT-LLM backend setup](/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/overview).

    ```bash
    bash examples/backends/trtllm/launch/agg_multimodal.sh
    ```

    The launcher serves `Qwen/Qwen2-VL-7B-Instruct` with the engine
    configuration in `examples/backends/trtllm/engine_configs/qwen2-vl-7b-instruct/agg.yaml`.
    Set `"model": "Qwen/Qwen2-VL-7B-Instruct"` in the request below. See
    [TensorRT-LLM Multimodal](/dynamo/dev/knowledge-base/modular-components/backends/tensor-rt-llm/tensorrt-llm-multimodal)
    for Llama 4, disaggregated layouts, and pre-computed embeddings.
  </Tab>
</Tabs>

## Send an Image Request

In another terminal, send a chat request with an `image_url` content part:

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-VL-2B-Instruct",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"}}
      ]
    }],
    "max_tokens": 64
  }'
```

To send a local file, base64-encode it as a `data:` URL:
`"url": "data:image/jpeg;base64,<encoded bytes>"`.

## Media URL Policy

Workers fetch media URLs under a default-deny policy:

- `https://` and `data:` sources are allowed.
- Plain `http://` URLs and hostnames that resolve to private or loopback
  addresses are refused. Set `DYN_MM_ALLOW_INTERNAL=1` on the worker that loads
  the media to fetch from the cluster's internal network.
- A worker that reaches the internet through an HTTP proxy must set
  `DYN_MM_TRUST_EGRESS_PROXY=1`. Without it, a fetch that would go through the
  proxy is refused. Direct fetches, including hosts in `NO_PROXY`, are unaffected.
- Each remote download is capped at `DYN_MM_MAX_FILE_SIZE_MB` (default `64`).
  TensorRT-LLM uses its own `--max-file-size-mb` option instead.

<Warning>
`DYN_MM_ALLOW_INTERNAL=1` lets request authors make the worker fetch internal
URLs. Enable it only when requests come from trusted clients.
</Warning>

## Next Steps

- [Custom Vision Encoders](/dynamo/dev/image-understanding/custom-vision-encoders): run your own vision tower in a vLLM worker.

## Performance Optimizations

Once requests work, you can cut latency and raise throughput for image
workloads. [VLM Performance Optimization](/dynamo/dev/image-understanding/vlm-performance-optimization) shows
where a request spends its time and when each feature helps:

- [Multimodal KV routing](/dynamo/dev/image-understanding/vlm-performance-optimization/multimodal-kv-routing): send repeated images to the worker that already holds their KV cache.
- [Frontend media decoding](/dynamo/dev/image-understanding/vlm-performance-optimization/frontend-media-decoding): decode images in the Rust frontend.
- [Shared image cache](/dynamo/dev/image-understanding/vlm-performance-optimization/shared-image-cache): reuse downloaded image bytes across workers.
- [Embedding cache](/dynamo/dev/image-understanding/vlm-performance-optimization/embedding-cache): skip the vision encoder for images it has seen.
- [Encoder disaggregation](/dynamo/dev/image-understanding/vlm-performance-optimization/encoder-disaggregation): scale vision encoders separately from LLM workers.