> This page is for version 1.0.0 (Latest) (default). > For other versions, use one of these documentation indexes: > - 1.0.0 (Latest) (default): https://docs.nvidia.com/nim/alpamayo/1.0.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nim/alpamayo/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/alpamayo/_mcp/server. # Sampling Control Alpamayo1.5 NIM exposes request parameters for controlling VLM rollout, trajectory sampling, and optional navigation text. ## Trajectory Inputs Trajectory requests require compressed camera images and ego-motion history. The default layout is 16 images: four timesteps for each of four cameras. When `camera_order` is omitted, the NIM assumes this default camera-major order: ```text cross_left_t0, cross_left_t1, cross_left_t2, cross_left_t3, front_wide_t0, front_wide_t1, front_wide_t2, front_wide_t3, cross_right_t0, cross_right_t1, cross_right_t2, cross_right_t3, front_tele_t0, front_tele_t1, front_tele_t2, front_tele_t3 ``` `t0` is the oldest frame, approximately `t-0.3s`. `t3` is the current frame. Reduced image layouts are supported. For any trajectory request that does not use the default 16-image layout, include both `camera_order` and `num_frames_per_camera`. Images must still be ordered camera-major: all timesteps for the first camera, then all timesteps for the next camera. For example: ```json { "camera_order": ["front_wide"], "num_frames_per_camera": 4 } ``` This layout expects `front_wide_t0` through `front_wide_t3`. A layout such as `camera_order=["cross_left", "front_wide", "cross_right", "front_tele"]` and `num_frames_per_camera=1` expects one current-frame image for each camera. If a trajectory request sends fewer or more than 16 images without `camera_order`, request-model validation returns HTTP 422 because the NIM cannot infer the image grouping. Camera names can be full Alpamayo camera names, such as `camera_front_wide_120fov`, or OSS short names, such as `front_wide`. The ego-motion history length is independent of the image layout. Every trajectory request, including reduced and single-image requests, requires all 16 history entries. Do not shorten the history to match the number of images. Use JPEG input images when possible. JPEG payloads are smaller than PNG and use the fastest image decode path in the NIM. ## VQA Inputs Visual question answering accepts one or more compressed images and preserves the image order sent by the client. VQA requests do not require ego-motion. Use JPEG images when possible. Common benchmark-style layouts such as one image, three images, five frames, and six camera views are accepted. ## Sampling Parameters | Name | Type | Default | Required | Description | | ----------------------- | ------- | ------------------------------------- | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `num_traj_samples` | integer | `1` | No | Number of trajectory samples to generate. The value must be listed in `NIM_ALPAMAYO_TRAJ_SAMPLES`. | | `nav_text` | string | unset | No | Optional route context for trajectory generation. It is advisory model input, not a deterministic vehicle-control command or a safety control. | | `temperature` | float | `0.6` | No | VLM sampling temperature. | | `top_p` | float | `0.98` | No | VLM nucleus sampling top-p. | | `top_k` | integer | unset | No | Optional VLM top-k sampling limit. Use `1` for top-1 decoding. | | `max_tokens` | integer | `256` | No | Maximum VLM generation tokens. | | `egomotion` | object | unset | Yes (trajectory); No (VQA) | Ego-motion payload containing `ego_history_xyz` with shape `[16, 3]` and `ego_history_rot` with shape `[16, 4]` (each quaternion `[w, x, y, z]`) or `[16, 3, 3]` for rotation matrices. Other history lengths are rejected before inference. | | `images` | array | unset | Yes | Compressed camera images as base64 strings, data URLs, or compressed image bytes depending on the API used. The default trajectory layout is 16 images. Non-default trajectory layouts require `camera_order` and `num_frames_per_camera`. | | `camera_order` | array | unset | No | Camera names matching the camera-major image order. Full Alpamayo camera names and OSS short names are accepted. Required for trajectory requests when image count is not 16. | | `num_frames_per_camera` | integer | unset; the default layout implies `4` | Required for non-default layouts | Number of temporal frames supplied for each camera. Omit this field and `camera_order` for the default 16-image layout. Set both fields explicitly for every reduced layout. | | `seed` | integer | unset | No | Optional per-request sampling seed. When omitted, the container default seed is used. | ## Reproducibility HTTP and gRPC requests can include `seed` to initialize sampling state. When `seed` is omitted, the container-level `NIM_ALPAMAYO_SEED` value is used. For repeatable K=1 trajectory inference in the same deployment, use the same input with `num_traj_samples=1`, a fixed `seed`, `top_k=1`, and `top_p=1`: ```json { "num_traj_samples": 1, "seed": 42, "top_k": 1, "top_p": 1 } ``` The seed controls diffusion sampling. `top_k=1` requests top-1 VLM decoding, and `top_p=1` leaves nucleus sampling unrestricted so it does not filter the top-1 token. ## K Values The active K values are controlled by `NIM_ALPAMAYO_TRAJ_SAMPLES`. Configure only the K values your deployment needs. Use an explicit comma-separated list such as `1,2,3` or `1,10`. Each configured value is warmed during startup and retained for CUDA graph performance, so broad K lists can increase startup time and GPU memory usage. Requests outside the configured list return a client error. For example, `num_traj_samples=2` when `NIM_ALPAMAYO_TRAJ_SAMPLES=1,10` returns an error that the profile supports only those values. For the full profile matrix, refer to [Support Matrix](/nim/alpamayo/support-matrix). ## Trajectory Metrics Trajectory responses include a `metrics` object with trajectory-only kinematic metrics. The primary comfort metrics are scalar pass rates in `[0, 1]` across the generated trajectory samples: - `comfort_lat_accel` - `comfort_lon_accel` - `comfort_lon_jerk` - `comfort_yaw_accel` - `comfort_yaw_rate` These metrics are computed from the generated trajectory and predicted ego rotations. Evaluation-only metrics that require ground truth or map geometry are not part of the normal inference response. The object also reports its schema `version`, `time_step_s`, comfort thresholds, aggregate `min`/`mean`/`max` kinematic summaries, and a `per_sample` breakdown. Refer to [API Reference](/nim/alpamayo/api-reference) for the full field list and units. The response includes `pred_rotations` as rotation matrices with shape `[K, num_points, 3, 3]`. This matches the OSS `pred_rot` output after squeezing the batch and trajectory-set dimensions.