Quickstart Guide

Start by completing the Cosmos 3 Installation Guide, including access to the gated model repository you want to use. Cosmos 3 offers separate Generator and Reasoner paths, so select the integration that fits your workflow.

Select an integration

Workload	Integration	When to use it	Instructions
Generator research	Diffusers	Python-first development and model experimentation	Cosmos repository: Generator with Diffusers
Generator turnkey deployment	NIM	Prebuilt Generator container for text-to-video and image-to-video	Cosmos repository: Generator with NIM
Generator API serving	vLLM-Omni or SGLang	OpenAI-compatible serving for supported multimodal generation	vLLM-Omni Cosmos3-Nano recipe or Cosmos repository: Generator with SGLang
Reasoner research	Transformers	Python-first text, image, and video reasoning	Cosmos repository: Reasoner with Transformers
Reasoner production inference	vLLM or TensorRT-LLM	OpenAI-compatible text responses from image and video inputs	Cosmos repository: Reasoner with vLLM or TensorRT-LLM cookbook
Reasoner turnkey deployment	NIM	Prebuilt, optimized OpenAI-compatible Reasoner container	Cosmos repository: Reasoner with NIM

Run a Reasoner example with Transformers

The following Python-first example runs image captioning with the nvidia/Cosmos3-Nano Reasoner. Create an environment and install the packages required by the source quickstart:

uv venv --python 3.13 --seed --managed-python
source .venv/bin/activate
uv pip install --torch-backend=auto \
  accelerate \
  av \
  pillow \
  "safetensors>=0.8.0" \
  torch \
  "torchvision==0.25.0" \
  "transformers>=5.11.0"

Save an input image as example.jpg, then run:

from pathlib import Path

import torch
from transformers import AutoProcessor, Cosmos3OmniForConditionalGeneration

model_id = "nvidia/Cosmos3-Nano"
image_path = Path("example.jpg").resolve()

processor = AutoProcessor.from_pretrained(model_id)
model = Cosmos3OmniForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "path": str(image_path)},
            {"type": "text", "text": "Caption the image in detail."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, torch.bfloat16)

generated_ids = model.generate(**inputs, do_sample=False, max_new_tokens=512)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(output[0])

For the larger Reasoner, change model_id to nvidia/Cosmos3-Super. With Accelerate installed, device_map="auto" can shard the model across multiple GPUs.

Next steps

The Cosmos repository and its Cosmos 3 cookbooks contain the maintained, integration-specific setup and usage instructions. Refer to the Model Matrix to choose the appropriate Cosmos 3 model.