Skip to navigation
Image Understanding

Image Understanding

Serve vision language models that answer questions about images
View as Markdown

Dynamo serves vision language models (VLMs) through the OpenAI-compatible /v1/chat/completions endpoint. A request carries one or more image_url content parts next to its text, and the worker runs the model’s vision encoder before prefill.

This page gets one image request working on each backend. To generate images instead, see Text to Image. To speed up image workloads, see VLM Performance Optimization.

Support Matrix

InputvLLMSGLangTensorRT-LLM
https:// image URLYesYesYes
data: (base64) imageYesYesYes
Pre-computed embeddings (.safetensors)NoNoYes

Plain http:// URLs and hosts that resolve to private or loopback addresses are refused by default. See Media URL Policy.

Set up the backend

The vLLM and TensorRT-LLM launchers below start the worker with --enable-multimodal. The SGLang aggregated launcher (agg_vision.sh) does not pass it, while the SGLang disaggregated launchers do. See SGLang Multimodal for the flags each worker role needs.

Run the commands from the root of a Dynamo repository checkout. Each launcher starts a frontend on port 8000 and one aggregated worker on GPU 0.

Requires a working vLLM backend setup.

bash examples/backends/vllm/launch/agg_multimodal.sh --model Qwen/Qwen3-VL-2B-Instruct

The launcher defaults to Qwen/Qwen3-VL-30B-A3B-Instruct-FP8 when --model is omitted. For XPU devices, use launch/xpu/agg_multimodal_xpu.sh. See vLLM Multimodal for disaggregated layouts, LoRA adapters, and supported models.

Send an Image Request

In another terminal, send a chat request with an image_url content part:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-VL-2B-Instruct",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"}}
      ]
    }],
    "max_tokens": 64
  }'

To send a local file, base64-encode it as a data: URL: "url": "data:image/jpeg;base64,<encoded bytes>".

Media URL Policy

Workers fetch media URLs under a default-deny policy:

  • https:// and data: sources are allowed.
  • Plain http:// URLs and hostnames that resolve to private or loopback addresses are refused. Set DYN_MM_ALLOW_INTERNAL=1 on the worker that loads the media to fetch from the cluster’s internal network.
  • A worker that reaches the internet through an HTTP proxy must set DYN_MM_TRUST_EGRESS_PROXY=1. Without it, a fetch that would go through the proxy is refused. Direct fetches, including hosts in NO_PROXY, are unaffected.
  • Each remote download is capped at DYN_MM_MAX_FILE_SIZE_MB (default 64). TensorRT-LLM uses its own --max-file-size-mb option instead.

DYN_MM_ALLOW_INTERNAL=1 lets request authors make the worker fetch internal URLs. Enable it only when requests come from trusted clients.

Next Steps

Performance Optimizations

Once requests work, you can cut latency and raise throughput for image workloads. VLM Performance Optimization shows where a request spends its time and when each feature helps: