Skip to navigation

Overview

View as Markdown

Alpamayo1.5 NIM serves nvidia/alpamayo1.5 for trajectory prediction and visual question answering. It accepts camera images, ego-motion history, optional navigation text, and optional questions. For trajectory requests, it returns sampled future trajectories.

Use nvidia/alpamayo1.5 as the model ID in API requests. The NGC resource and container image use the packaging name alpamayo-1-5-10b; the two names identify the served model and its distributable container, respectively.

The NIM exposes HTTP APIs for trajectory generation and VQA. It also exposes a binary gRPC API for trajectory generation.

Architecture

At a high level, Alpamayo1.5 combines vision-language understanding with a trajectory-generation model:

  • A vision-language model processes the multi-camera temporal context, ego-motion history, navigation text, and VQA questions.
  • For trajectory generation, the trajectory expert and diffusion head use that context to sample future ego trajectories.
  • For VQA, the vision-language path returns a natural-language answer.

Precision Selection

The NIM automatically selects a compatible runtime profile for the visible GPU. To request a specific precision, set NIM_PRECISION to bf16, fp8, or w4a16. The runtime maps that precision to the matching manifest profile and then applies GPU capability, memory, and K-value gates.

The active trajectory count defaults to K=1. Set NIM_ALPAMAYO_TRAJ_SAMPLES to the exact K values that the container should warm and serve, for example 1,2,3 or 1,10. For the full GPU and profile matrix, refer to Support Matrix.

APIs

The container includes the generated OpenAPI specification at /opt/nim/api_spec.yaml. The spec covers the health and metadata routes plus the Alpamayo HTTP inference routes:

  • POST /v1/infer for native Alpamayo trajectory generation, VQA, or both outputs in one request.
  • POST /v1/chat/completions for OpenAI-compatible trajectory generation.
  • POST /v1/vqa for OpenAI-compatible visual question answering.

For request examples and the embedded OpenAPI schema, refer to API Reference.