Is this page helpful?

Introduction#

Alpamayo1.5 NIM serves nvidia/alpamayo1.5 for trajectory prediction and visual question answering. It accepts camera images, ego-motion history, optional navigation text, and optional questions. For trajectory requests, it returns sampled future trajectories.

Use nvidia/alpamayo1.5 as the model ID in API requests. The NGC resource and container image use the packaging name alpamayo-1-5-10b; the two names identify the served model and its distributable container, respectively.

The NIM exposes HTTP APIs for trajectory generation and VQA. It also exposes a binary gRPC API for trajectory generation.

Architecture#

At a high level, Alpamayo1.5 combines vision-language understanding with a trajectory-generation model:

  • A vision-language model processes the multi-camera temporal context, ego-motion history, navigation text, and VQA questions.

  • For trajectory generation, the trajectory expert and diffusion head use that context to sample future ego trajectories.

  • For VQA, the vision-language path returns a natural-language answer.

Precision Selection#

The NIM automatically selects a compatible runtime profile for the visible GPU. To request a specific precision, set NIM_PRECISION to bf16, fp8, or w4a16. The runtime maps that precision to the matching manifest profile and then applies GPU capability, memory, and K-value gates.

The active trajectory count defaults to K=1. Set NIM_ALPAMAYO_TRAJ_SAMPLES to the exact K values that the container should warm and serve, for example 1,2,3 or 1,10. For the full GPU and profile matrix, refer to Support Matrix.

APIs#

The container includes the generated OpenAPI specification at /opt/nim/api_spec.yaml. The spec covers the health and metadata routes plus the Alpamayo HTTP inference routes:

  • POST /v1/infer for native Alpamayo trajectory generation, VQA, or both outputs in one request.

  • POST /v1/chat/completions for OpenAI-compatible trajectory generation.

  • POST /v1/vqa for OpenAI-compatible visual question answering.

For request examples and the embedded OpenAPI schema, refer to API Reference.