SO-101 Reach Task - Part 3: Deploying the Policy
Deploy an SO-101 reach policy with Isaac OS: configure the application, check it without hardware, calibrate the arm and run closed-loop control. Use the default policy the build downloads, the policy from Part 1, or a compatible ONNX export. Run it against a simulated arm first with Part 2: Simulation.
Prerequisites
Before starting, ensure you have:
-
Isaac OS repo cloned and Docker image built — see Getting Started.
-
A CUDA-capable GPU. Every run mode builds a TensorRT engine from the policy and refuses to start without a device.
-
SO-101 hardware — a calibrated arm connected via USB.
-
Follow LeRobot’s SO 101 setup for assembly and calibration.
-
Grant your user access to the serial port:
./isaac_osruns the container as your user without thedialoutgroup, so without this ACL the application cannot open the port. The ACL is lost when you reboot or replug the robot, so re-run the command then. If the robot enumerates as another node, such as/dev/ttyACM1, adjust the path andreal_robot.serial_portto match.
-
-
Optionally, a policy you trained in Part 1, exported as ONNX with a float32
obsinput of shape[1, 20]and a float32actionsoutput of shape[1, 4]. The application verifies both tensors against the engine at startup and refuses to run otherwise. Without one, it runs the default policy the build downloads.The runtime does not read the IO descriptor; you copy its values into the YAML in Configuration.
How the Application Works
- State source — reads Feetech positions and velocities at 100 Hz; mock mode emits a constant pose.
- Policy —
So101ReachFusedOpcomputes fingertip FK and the observation, then runs TensorRT at 50 Hz. - Motion control — bounds each policy delta against the freshest measurement on every state.
- Command sink — writes servo positions; mock mode validates each command.
The controller runs on every state and holds the latest policy target for up to
motion_control.target_expiry, decoupling the 50 Hz policy from the 100 Hz commands.
The architecture reference
covers partitions, connections, and the staleness and fault rules.
Building the Application
The build downloads the default policy model.
To list the run modes:
Configuration
All tunable parameters live in
applications/so101_reach/config/so101_reach_config.yaml.
The binaries take --config-path and read the file before the graph
compiles. There is no --override flag, so experiments are made on a
copy of the YAML rather than by overriding keys on the command line.
Policy
To deploy your Part 1 policy, set policy.onnx_path to your file outside
data/grip_reach_delta_policy_onnx/ and copy the four IO-descriptor values:
Robot Hardware
serial_port— USB device for the Feetech bus.calibration_file— path to the LeRobot calibration JSON (see calibration below). The shipped placeholder is rejected at startup.
Keep position_i_gain non-zero. With LeRobot’s default I = 0,
shoulder_lift and elbow_flex cannot lift the arm’s weight, which looks like a bad policy.
The configuration reference explains every key, and the hardware notes explain the servo gains and speed.
Calibration
The SO-101 requires a LeRobot calibration JSON file that maps motor IDs to joint names and homing offsets. The expected format:
Motor IDs must be contiguous starting from 1. The joint order must
match: shoulder_pan, shoulder_lift, elbow_flex,
wrist_flex, wrist_roll, gripper.
LeRobot writes the file to
~/.cache/huggingface/lerobot/calibration/robots/so_follower/<robot.id>.json.
The directory is so_follower, not so101_follower: LeRobot 0.6.1
shares one calibration directory across the SO-100 and SO-101 followers.
Pose each joint at true mid-travel before pressing Enter during calibration, and hold the gripper half-open. Calibrate before running the application: torque stays enabled after the application exits, and a torqued joint cannot be hand-posed, so power-cycle the arm before recalibrating.
Since the application runs inside Docker, host paths are not visible by
default, and the real_robot keys are read straight from the YAML.
Copy the config, point it at the mounted calibration file, and mount the
directory.
Copy the config:
Then set real_robot.calibration_file in the copy to the absolute
mounted path. ./isaac_os sets HOME=/workspace/isaac_os inside the
container, so $HOME there resolves to the repository rather than your
home directory.
Mount the calibration directory:
$HOME inside the -v flag is fine: the host shell expands both
sides before Docker sees them.
Validating Without Hardware
mock mode runs the same policy and motion controller against a
synthetic state source, so it exercises the temporal contracts, the
TensorRT engine, and the command bounds before the arm is involved:
Success criterion: the command exits with status 0. A failed run logs
The SO-101 TensorRT mock graph did not produce valid joint commands.
The default run stops after 100 commands — about one second at 100 Hz — and gives up if they do not arrive within 30 s. For a longer run:
The mock state source publishes a constant pose and never integrates
the commands it receives, so the fingertip never approaches a goal.
Any run longer than policy.goal_hold_time logs goal timeouts by
design. Mock mode validates plumbing, not reach behavior.
To run the same policy against a simulated arm that responds to commands, follow Part 2: Simulation.
Running on Real Hardware
Safety Checklist
Before running:
- Clear 1 m radius around the robot.
- Verify joints can move freely.
- Have an emergency stop ready (power disconnect).
- Ready to kill the process (Ctrl+C).
Keep the workspace clear for the first homing motion in particular:
home_position puts wrist_flex at 1.5708 rad, so the arm moves
there before the policy takes over.
Motors stay in torque mode after the application exits, so the arm does not free-fall. Do not manually move joints while torque is enabled. Unplug the robot before repositioning.
Running
With the config copied and the calibration directory mounted:
To stop after a fixed number of commands instead of running until Ctrl+C:
On a run with a cold engine, the arm homes and then holds still while TensorRT builds the engine.
Read the Result
Filter the log for goal outcomes:
Success criterion: the arm moves toward each goal, and the log shows Reached goal
lines. A run that lasts through the whole sequence also logs a Completed all 9 goals
line with the number achieved; the 3,000-command run can end before that line.
❓ What happens during startup and shutdown?
Startup order matters for bring-up. The YAML is decoded and validated and the graph is compiled before the driver opens the serial port, so a bad config or an unbuildable graph fails while the arm is still idle. The driver then applies calibration, PID, and gripper protection, enables torque, and homes the arm.
The TensorRT engine is built after that, when the graph starts. A run with a cold engine therefore homes the arm and then holds it in place for the length of the engine build, and a missing or malformed ONNX is reported at that point rather than before homing.
On shutdown, the command sink writes the present positions back as the goal. Every command
is measured + delta — a position the arm has not reached yet — so closing the serial
port alone would leave the servos driving toward the last target after the process exits.
Troubleshooting
⚠️ Startup and motion problems
Summary
In this part you learned how to:
- Configure the policy and hardware keys.
- Validate without hardware —
mockmode as a contract and engine check. - Run on hardware and judge the result — configure calibration, confirm the arm moves toward its goals, and read the goal logs.
Next Steps
- Part 4: Benchmark — measure the pipeline’s latency and cadence with Nsight Systems.