Quickstart Guide#
This guide provides instructions on running inference with the Cosmos-Transfer2.5/general model.
Note
Ensure you have completed the steps in the Transfer2.5 Installation Guide before running inference.
Example Inference Command#
Individual control variants can be run on a single GPU:
python examples/inference.py --params_file assets/robot_example/depth/robot_depth_spec.json
For multi-GPU inference on a single control, or to run multiple control variants, use torchrun:
export NUM_GPUS=8
torchrun --nproc_per_node=$NUM_GPUS --master_port=12341 -m examples.inference --params_file assets/robot_example/vis/robot_vis_spec.json --num_gpus=$NUM_GPUS
Example Parameter Files#
An example parameter file for each individual control variant is provided, along with a multi-control variant:
Variant |
Parameter File |
---|---|
Depth |
|
Edge |
|
Segmentation |
|
Blur |
|
Multi-control |
|
Parameters can be specified as follows:
{
// Path to the prompt file, use "prompt" to directly specify the prompt
"prompt_path": "assets/robot_example/robot_prompt.json",
// Directory to save the generated video
"output_dir": "outputs/robot_multicontrol",
// Path to the input video
"video_path": "assets/robot_example/robot_input.mp4",
// Inference settings
"guidance": 3,
// Depth control settings
"depth": {
// Path to the control video
// For "vis" and "edge", if a control is not provided, it will be computed on the fly.
"control_path": "assets/robot_example/depth/robot_depth.mp4",
// Control weight for the depth control
"control_weight": 0.5
},
// Edge control settings
"edge": {
// Path to the control video
"control_path": "assets/robot_example/edge/robot_edge.mp4",
// Default control weight of 1.0 for edge control
},
// Seg control settings
"seg": {
// Path to the control video
"control_path": "assets/robot_example/seg/robot_seg.mp4",
// Control weight for the seg control
"control_weight": 1.0
},
// Blur control settings
"vis":{
// Control video computed on the fly
"control_weight": 0.5
}
}
Example Output#
The following video shows output from a multiple control variant:
Next Steps#
Refer to the :ref:Transfer2.5 Model Reference <transfer2.5-model-reference>
page for more information on running inference with the Auto Multiview model. If you’re ready to start post-training, refer to the :ref:Transfer2.5 Post-Training Guides <transfer2.5-post-training-guides>
page.