Is this page helpful?

Bring Your Own Checkpoint#

Alpamayo1.5 NIM supports Bring Your Own Checkpoint (BYOC) with NIM_FT_CHECKPOINT.

Checkpoint Requirements#

The checkpoint must be a full Alpamayo Hugging Face checkpoint directory, or a workspace-style directory that contains alpamayo/config.json.

Single-file training deltas, LoRA adapters, and expert-only checkpoints are not supported until they are merged or converted into a full Alpamayo Hugging Face checkpoint.

Launch with BYOC#

For a checkpoint downloaded from Hugging Face, use --local-dir to create a complete, stable host directory before mounting it. The default cache snapshot printed by hf download can contain relative links to blob files outside the snapshot. Mounting only that snapshot can leave the links unreadable inside the container, causing startup to report that it is not a full Alpamayo checkpoint.

export IMG_NAME=nvcr.io/nim/nvidia/alpamayo1.5:1.0.0
export LOCAL_NIM_CACHE=/tmp/alpamayo-nim-cache
python3 -m pip install -U huggingface_hub
export ALPAMAYO_HF_DIR="$PWD/alpamayo-hf"
mkdir -p "$LOCAL_NIM_CACHE"
chmod 777 "$LOCAL_NIM_CACHE"
hf download nvidia/Alpamayo-1.5-10B --local-dir "$ALPAMAYO_HF_DIR"
test -f "$ALPAMAYO_HF_DIR/alpamayo/config.json" || \
   test -f "$ALPAMAYO_HF_DIR/config.json"

docker run -it --rm --name alpamayo1.5-byoc \
   --runtime=nvidia \
   --gpus '"device=0"' \
   --shm-size=16g \
   --ulimit memlock=-1 \
   --ulimit stack=67108864 \
   -e NGC_API_KEY=$NGC_API_KEY \
   -e NIM_FT_CHECKPOINT=/byoc/alpamayo \
   -v "$ALPAMAYO_HF_DIR:/byoc/alpamayo:ro" \
   -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
   -p 8000:8000 \
   -p 50051:50051 \
   $IMG_NAME

If the default cache snapshot was already downloaded, create a complete local copy by dereferencing its links before mounting it:

export ALPAMAYO_HF_SNAPSHOT=$(hf download nvidia/Alpamayo-1.5-10B)
export ALPAMAYO_HF_DIR="$PWD/alpamayo-hf"
mkdir -p "$ALPAMAYO_HF_DIR"
cp -aL "$ALPAMAYO_HF_SNAPSHOT"/. "$ALPAMAYO_HF_DIR"/

Hugging Face can also create hidden download metadata under $ALPAMAYO_HF_DIR/.cache with permissions inherited from the host user. Some older NIM images included matching files under that directory while computing the BYOC checkpoint cache key. In that case, container UID 1000 can fail with Permission denied while reading the metadata even though the actual checkpoint files are readable.

The hidden metadata is not part of the checkpoint. When using an affected image, remove only that directory and restart the container:

rm -rf "$ALPAMAYO_HF_DIR/.cache"

Use the same mount layout for any existing Alpamayo directory that contains config.json at its root. A workspace containing alpamayo/config.json can instead be mounted at /byoc with NIM_FT_CHECKPOINT=/byoc.

The checkpoint should include or reference the tokenizer and processor files needed by Alpamayo. Startup uses a BYOC-local cosmos-reason2 directory when present, a local vlm_name_or_path when it resolves to a mounted directory, or the packaged cosmos-reason2 processor files.

Cache Behavior#

Startup creates cache-isolated patched model artifacts under $NIM_CACHE_PATH/alpamayo-trtllm/byoc/<checkpoint_hash>/. BF16 profiles load the BYOC checkpoint directly. FP8 and W4A16 profiles create or reuse a BYOC-specific ModelOpt VLM export under the same cache key; they do not reuse the prebuilt NGC ModelOpt VLM checkpoints because those correspond to the default weights.

Keep /opt/nim/.cache mounted if you want to avoid re-running BYOC quantization on restart.

Calibration Dataset#

For BYOC FP8 and W4A16 profiles, mount calibration scenes into the container and set NIM_ALPAMAYO_CALIBRATION_SAMPLE_LIST. The value is a comma-separated list of text manifest files, not a list of images. Each manifest file must contain one Alpamayo scene directory per line. Blank lines and lines beginning with # are ignored.

For example, create a manifest file:

/calib/scene_0001
/calib/scene_0002
# /calib/scene_disabled
/calib/scene_0003

Then mount the calibration data and pass the manifest path:

docker run ... \
   -v /host/calib:/calib:ro \
   -e NIM_PRECISION=fp8 \
   -e NIM_FT_CHECKPOINT=/byoc/alpamayo \
   -e NIM_ALPAMAYO_CALIBRATION_SAMPLE_LIST=/calib/alpamayo_scenes.txt

When this variable is unset, startup uses the packaged sample scene.

Metadata#

GET /v1/models reports checkpoint as default or byoc and includes a short checkpoint_hash for BYOC runs.