Bring Your Own Checkpoint#
Use this page to replace the selected Generator or Reasoner checkpoint while preserving the Cosmos3 NIM server, profile selection, and runtime contract.
Supported Boundary#
NIM_MODEL_PATH is the shared checkpoint-source variable:
Runtime |
Accepted Source |
What the NIM Still Provides |
|---|---|---|
Generator |
Absolute local directory or |
Server, profile, and Generator guardrail artifacts |
Reasoner |
Absolute local directory or |
Server, selected runtime layout, and profile compatibility policy |
Nano or Super Reasoner DFlash draft |
Absolute local directory only |
Primary Reasoner checkpoint, server, and profile policy |
Generator Checkpoint#
Expected Layout#
The current Generator path expects this structural shape:
/byoc/cosmos3/
├── transformer/
│ ├── config.json
│ └── <weight shards>
├── vae/
├── scheduler/
└── model_index.json
The NIM infers Edge, Nano, or Super and the precision from
transformer/config.json and checks them against the selected profile. Set
NIM_MODEL_VARIANT to select a specialist Generator contract. The only Edge
Generator contract is the BF16 edge-droid policy.
Launch#
Add these options to the command in
Launch Generator, with BYOC_CHECKPOINT set
to the host checkpoint directory:
-e NIM_MODEL_PATH=/byoc/cosmos3 \
-v "$BYOC_CHECKPOINT:/byoc/cosmos3:ro"
Choose a variant, precision, and objective that match the checkpoint. The NIM
still downloads Generator guardrail artifacts, so the cache must be writable,
the NIM can still need NVIDIA NGC access, and NIM_DISABLE_MODEL_DOWNLOAD=1 is rejected.
Hugging Face Source#
The Generator also accepts hf://owner/repository[:revision], downloaded into
the NIM cache. The repository root must contain the layout above, and the
inferred variant and precision must match the selected profile:
-e NIM_MODEL_PATH='hf://owner/repository:revision' \
-e HF_TOKEN \
-e NIM_MODEL_VARIANT=nano \
-e NIM_PRECISION=fp8
Omit :revision to use main. Pin a commit for reproducible deployments. Pass
HF_TOKEN only for a private repository. Keep NGC_API_KEY and a writable
cache for the guardrail artifacts.
Reasoner Checkpoint#
Local Directory#
A local Reasoner checkpoint must use an absolute in-container path and contain at least the following:
A supported Cosmos3 Reasoner or Cosmos3 Omni
config.jsonSafetensors weights or a valid safetensors index and all referenced shards
Tokenizer files
Processor or preprocessor configuration
The NIM infers edge, nano, or super, the precision, and the Reasoner or
Omni layout, and then selects a compatible profile. The precision is BF16 or FP8
for Edge, and Nano and Super can also use NVFP4. An explicit
NIM_MODEL_VARIANT, NIM_PRECISION, or NIM_MODEL_PROFILE must agree with the
checkpoint. Mount it read-only and select the Reasoner:
-e NIM_MODEL_TYPE=reasoner \
-e NIM_MODEL_PATH=/byoc/cosmos3-reasoner \
-v "$BYOC_CHECKPOINT:/byoc/cosmos3-reasoner:ro"
With a local checkpoint, the NIM sets NIM_DISABLE_MODEL_DOWNLOAD=1
automatically, so it does not download profile artifacts. With NIM_MODEL_PATH, the NIM does not download the
profile’s DFlash draft, which Nano and Super Reasoner enable by default. Mount a
matching Nano or Super draft through
NIM_DFLASH_MODEL_PATH (absolute local path, no hf://), or set
NIM_USE_DFLASH=0 to run target-only. Edge has no DFlash. It defaults to
NIM_USE_DFLASH=0 and rejects 1. The draft needs only:
/byoc/cosmos3-dflash/
├── config.json
└── model.safetensors
Add these options to the Reasoner docker run command, with
DFLASH_CHECKPOINT set to the host draft directory:
-e NIM_USE_DFLASH=1 \
-e NIM_DFLASH_MODEL_PATH=/byoc/cosmos3-dflash \
-v "$DFLASH_CHECKPOINT:/byoc/cosmos3-dflash:ro"
To train your own matching Cosmos3-Nano DFlash draft on a custom dataset, or after fine-tuning the Nano Reasoner target, follow the NVIDIA Model Optimizer DFlash training recipe.
Hugging Face Source#
The Reasoner also accepts:
-e NIM_MODEL_TYPE=reasoner \
-e NIM_MODEL_PATH='hf://owner/repository:revision' \
-e HF_TOKEN
Omit :revision to use main. Pass HF_TOKEN only when the repository
requires it, never in the URI or image. The snapshot is stored in the writable
NIM cache.
An hf:// source cannot be combined with NIM_DISABLE_MODEL_DOWNLOAD=1. For
offline use, pre-download the checkpoint and use an absolute local path. The
DFlash draft rule above still applies.
Verify the Active Checkpoint#
After readiness, inspect metadata:
curl -fsS http://localhost:8000/v1/metadata | python3 -m json.tool
The checkpoint field reports default for bundled artifacts or identifies the
configured source. Generator metadata also reports model_variant. Confirm the
selected profile, then compare a representative request with the checkpoint’s
validation baseline.
Common Failures#
Symptom |
Likely Cause |
Action |
|---|---|---|
Local path does not exist |
Host path was not mounted at the exact |
Align the read-only bind destination and environment path |
Permission denied |
Container cannot traverse or read the mount |
Fix host ownership or ACLs while keeping the checkpoint mount read-only |
Required file missing |
Checkpoint layout, tokenizer, processor, index, or weight shards are incomplete |
Compare with the required layout and export again |
Model variant or precision mismatch |
Explicit selector/profile disagrees with inferred checkpoint properties |
Select a compatible configuration or use a matching checkpoint |
Generator rejects disabled downloads |
Generator still needs NIM-provided guardrails |
Remove |
Hugging Face source rejected offline |
|
Use an absolute pre-downloaded local path |
Hugging Face authorization fails |
Token, repository ID, revision, network, or cache is invalid |
Check |
DFlash path is rejected |
The draft path is relative, uses |
Use an absolute local mount containing |
DFlash configuration is rejected |
DFlash is disabled, DFlash is enabled on Edge (including through |
For Edge, leave |
Metadata shows |
Override was omitted, rejected, or applied to another container |
Inspect launch environment, mounts, startup logs, and |
For broader startup, cache, GPU, and readiness diagnosis, refer to Operations.