Deploy a learned policy
A trained policy, such as a vision-language-action (VLA) model, only behaves as trained if it receives the same observations and its actions are read the same way. This guide pins down that model contract, picks an inference path, connects the policy and validates it in simulation before you use a physical robot.
This guide does not cover low-level controllers or task planners. To connect robot state and commands first, see Add a robot embodiment.
✨ Agent prompt: Deploy a learned policy
Check the model contract
A policy model receives observations and produces actions. Before you deploy an exported model, record these properties for each model file:
- Input and output tensor names, shapes and data types.
- Observation preprocessing: image crop, size, channel order, normalization, and the order of state values.
- Action meaning: joint order, units, and position or effort targets.
- Action timing: the number of actions in each prediction and the time between actions.
Matching file names, joint counts or horizons is not sufficient. Compare each
property with the application that runs the model. Export each model to ONNX.
InferenceOp also accepts prebuilt TensorRT engines. Keep each ONNX
external-data file at the path that its model references.
LEAPP exports: LEAPP writes ONNX files and a metadata YAML that describes the tensors and their semantics. Isaac OS does not load or interpret this YAML. Use it as the reference for the contract above, and write the Isaac OS configuration manually. Isaac ROS loads the LEAPP YAML to build its inference graph; see GR00T E2E deployment with Isaac ROS.
Select an inference path
Run the model with one of these operators:
- Upstream Holoscan
InferenceOp: use it when the models need TensorRT inference only. Put preprocessing and postprocessing in other operators.InferenceOpbuilds or loads a TensorRT engine for each ONNX model or prebuilt engine. It creates one port for each configured tensor, and it reads its model and tensor maps from C++ or YAML. It is not validated in an Isaac OS application yet. - Custom operator: use it when the policy needs preprocessing, a model chain
or action conversion that
InferenceOpdoes not do, or when you want to fuse these steps. Useisaac::TensorRTEnginefromcommon/tensorrt_engineto build and cache engines from ONNX.
For reference custom operators, see
operators/gr00t_n1_7 (five chained models, used
in the GR00T tutorials) and
operators/so101_reach (one model).
❓ Why does Isaac OS have its own GR00T operator?
Isaac OS built the GR00T operator before Holoscan SDK 5.0 improved the upstream
InferenceOp. GR00T N1.7 uses five chained models. The backbone output goes
into the action head, and the action head also needs a new noise input for each
inference. The GR00T operator fuses image preparation, joint gather, the five
TensorRT engines, action scatter, and output copies, and captures them in one
CUDA graph. See
GPU execution.
The Holoscan SDK 5.0 InferenceOp supports multiple models, CUDA Graphs and
several dispatch paths. When one InferenceOp runs chained models, it uses its
general dispatch path; see dispatch selection.
It is not evaluated as a replacement for the GR00T operator yet.
Connect the policy to the application
Connect operators that supply observations, such as cameras and joint state, to the policy inputs. Connect the policy actions to the operators that send robot commands. Add preprocessing and postprocessing operators between them when the inference operator does not do these steps.
Some differences need configuration changes only. Others need operator changes:
For the GR00T N1.7 operator, see the model export contract. It lists the model and application configuration files, the action chunk and joint message types, and how WBC combines the commands.
Validate the deployment
Do each check before you go to the next one:
- Run the policy without robot commands. Make sure that the models load and that the output shapes, value ranges and joint order are correct.
- Measure inference latency and action rate against the control budget. See Profile an Application and Optimize an Application.
- Run the task in simulation and observe the behavior.
- Run the task on the physical robot, with the robot in a safe area.
For GR00T N1.7 on G1, do these checks with the Simulation and Real robot tutorials.