NVIDIA Cosmos

Overview

• Introduction
  • Cosmos-Predict2
  • Cosmos-Predict1
    • Examples
      • Cosmos-Predict1-7B-Text2World-Multiview
      • Cosmos-Predict1-5B-Video2World
    • Getting Started Workflow
  • Cosmos-Transfer1
    • Examples
      • Cosmos-Transfer1-7B
      • Cosmos-Transfer1-7B [LiDAR|HDMap]
  • Cosmos-Reason1
• Release Notes
  • 1.1
  • 1.0
• Prerequisites
  • Hardware
    • Predict2
    • Predict1
    • Transfer1
    • Reason1
  • Software
• NVIDIA NIM for Cosmos

Cosmos-Predict2

• Model Matrix
  • Performance Benchmarks
  • Model Selection Guide
• Installation
  • Option 1: Conda Environment Setup
  • Next Steps
• Predict2 Quickstart Guide
  • Set up the Video2World Model
  • Generate a Video from Text Input
  • Next Steps
• Predict2 Model Reference
  • Model Types
  • Sample Commands
    • Text2Image
      • Downloading the Model Weights
      • Setting Prompts
      • Single Generation with 2B Model
      • Single Generation with 14B Model
    • Video2World
      • Downloading the Model Weights
      • Single Generation with 2B Model
      • Single Generation with 14B Model
  • Arguments
    • Common Parameters
    • Video2World-Specific Parameters
  • Prompting Guidelines
    • Prompt Example
    • Cosmos-Predict1-Prompt-Upsampler
    • Text2World
    • Video2World
  • Safety Features

Cosmos-Predict1

• Model Matrix
  • World Foundation Models
  • Tokenizers
• Installation
  • Cosmos-Predict1 Setup
  • Next Steps
• Diffusion
  • Diffusion Quickstart Guide
    • Set up the Diffusion Model
    • Generate a Video from Text Input
    • Next Steps
  • Diffusion Model Post-Training Guide
    • Setup
    • Preparing a Dataset
    • Post-Training
      • Text2World
        • Download the Sample Dataset
        • Preprocess the Data
        • Post-Train the Model
        • Test Inference
      • Video2World
        • Download the Sample Dataset
        • Preprocess the Data
        • Post-train the Model
        • Test Inference
      • Text2World-Multiview
        • Post-Train the Model
        • Test Inference
      • Video2World-Multiview
        • Post-Train the Model
        • Test Inference
  • Diffusion Model Reference
    • Model Types
    • Sample Commands
      • Text2World
        • Downloading the Model Weights
        • Single Generation
        • Single Generation with Model Offloading
        • Single Generation with Multi-GPU Inference
        • Batch Generation
      • Video2World
        • Downloading the Model Weights
        • Single Generation
        • Single Generation with Model Offloading
        • Single Generation with Multi-GPU Inference
        • Batch Generation
        • Batch Generation without Prompt Upsampling
      • Text2World-Multiview
        • Downloading the Model Weights
        • Multiview Prompts
        • Single Generation
        • Single Generation with Model Offloading
        • Single Generation with Multi-GPU Inference
        • Batch Generation
      • Video2World-Multiview
        • Downloading the Model Weights
        • Single Generation
        • Single Generation with Model Offloading
        • Single Generation with Multi-GPU Inference
        • Batch Generation
    • Arguments
      • Common Parameters
      • Video2World-Specific Parameters
    • Prompting Guidelines
      • Prompt Example
      • Cosmos-Predict1-Prompt-Upsampler
      • Text2World
      • Video2World
    • Safety Features
• Autoregressive
  • Autoregressive Quickstart Guide
    • Set up the Autoregressive Model
    • Generate a Video using Video Input
    • Next Steps
  • Autoregressive Post-Training Guide
    • Supported Models
    • Setup
    • Preparing a Dataset
    • Post-Training
      • Download the Sample Dataset
      • Post-Train the Model
        • Post-Train with a Single GPU (TP=1)
        • Post-train with 4 GPUs (TP = 4)
      • Test Inference
  • Autoregressive Model Reference
    • Model Types
    • Single and Batch Generation
      • Single Video
      • Batch Video
    • Text Prompt
    • Sample Commands
      • Base
        • Downloading the Model Weights
        • Single Generation
        • Single Generation with Multi-GPU Inference
        • Batch Generation
      • Video2World
        • Downloading the Model Weights
        • Single Generation
        • Single Generation with Multi-GPU Inference
        • Batch Generation
    • Arguments
      • Common Parameters
        • Base Specific Parameters
        • Video2World Specific Parameters
    • Safety Features
• Tokenizer
  • Tokenizer Inference Guide
    • Set up the Tokenizer
    • Tokenizer Overview
    • Examples
  • Tokenizer Post-Training Guide
    • Set up the Tokenizer
    • Post-training a Tokenizer
      • Download a Dataset
      • Post-train the Video Tokenizers
      • Inference with the Post-trained Model Checkpoint

Cosmos-Transfer1

• Installation
  • Cosmos-Transfer1 Setup
  • Next Steps
• Transfer Quickstart Guide
  • Set up Cosmos Transfer1
  • Generate a Visual Simulation from Source Video
• Transfer Model Reference
  • Sample Commands
    • Edge Detection ControlNet
      • Multi-GPU Inference
      • Inference with Prompt Upsampling
      • Batch Inference
    • Multimodal Control
      • Multimodal control with Spatiotemporal Control Map
    • Autonomous Vehicle Transfer
      • Multi-GPU Inference
      • Additional AV Toolkits
    • 4K Upscaling
      • Multi-GPU Inference
  • Arguments
  • ControlNet Specification
  • Prompting Guidelines
  • Safety Features

Cosmos-Reason1

• Installation
  • Cosmos-Reason1 Setup
  • Monitoring Setup
  • Next Steps
• Reason1 Training Guide
  • Training Configuration
  • Job recipes
  • GPU Requirements
  • Supervised Fine-Tuning (SFT)
  • Reinforcement Learning (RL)
  • Inference
  • Next Steps
• Reason1 Evaluation Guide
  • Download Benchmark Assets
  • Download Additional Video Clips (Optional)
  • Run Evaluation on Benchmarks
    • Configure Evaluation
    • Run Evaluation
  • Benchmark Scoring
    • About Evaluation
    • Usage

More Information

• Frequently Asked Questions
• Feedback and Support
• Cosmos Guardrail
  • Overview
    • Pre-Guard
    • Post-Guard
• License