SO-101 Reach Task - Part 2: Simulation
Run the SO-101 reach policy against a simulated arm in Isaac Sim before deploying it on hardware. You configure a policy, run a bounded closed-loop session, read the result, and check the loop timing and what the simulation does not show. Use the policy from Part 1 or the default policy the build downloads, then continue with Part 3: Deployment.
Prerequisites
Before starting, ensure you have:
- The Isaac OS repository cloned and its Docker image built. See Getting Started.
- A CUDA-capable GPU. The policy runs through TensorRT in every mode.
- Isaac Sim 6.2.0.1 set up with the Isaac Sim guide.
- Optionally, your Part 1 export: the
policy.onnxand its IO descriptor YAML.
What the Simulation Checks
The sim mode runs the same policy and motion controller as the real arm in
Part 3. Only the ends of the graph change:
joint states arrive from Isaac Sim over UCX, and joint commands go back the same way.
Simulated positions respond to commands, so the policy closes the loop and the fingertip
travels to each goal.
This makes the simulation a check of the deployment pipeline. It shows that your policy, configuration, joint order, frames and timing work together. It does not show that the real arm moves like the simulated one; see What the Simulation Does Not Show. A custom scene must keep the joint order, units and frames the policy was trained with; see the Isaac Sim reference.
Build and Check the Graph
Build the application and compile the simulation graph without starting a simulator:
The second command ends with
SO-101 reach simulation graph compiled. No simulator connection was opened.
Configure the Policy
As shipped, the application runs the default policy, so with it you can go straight to Start Isaac Sim.
Use your Part 1 policy
To run your own export, copy it to a path inside the repository, outside
data/grip_reach_delta_policy_onnx/, which the build overwrites:
In so101_reach_config.yaml, set policy.onnx_path to build/reach_policy/policy.onnx.
Then copy these values from your IO descriptor:
The application checks the tensors at startup. The policy needs a float32 obs input of
shape [1, 20] and a float32 actions output of shape [1, 4], and it must output joint
position deltas. See the
SO-101 reach configuration.
The first run with a new ONNX builds a TensorRT engine and caches it beside the model. Later runs load the cached engine.
Start Isaac Sim
Start the reach scene headless in one terminal:
The first start fills the simulator caches and can take several minutes. Later starts take under a minute. You can start the policy before Isaac Sim is ready: it waits for the connection.
Headless mode keeps each simulation step short, which matters for the timing check below.
To watch the arm move, start Isaac Sim without -- --headless instead.
Run a Bounded Session
Run the policy in a second terminal and keep a copy of the log:
--message-count=3000 stops the run after 3,000 joint states from the simulator. That is
about 50 seconds at the simulator’s usual speed, enough for two passes over the goal grid.
Isaac Sim starts playback when the application connects. When the run ends, the simulator resets the arm and waits for the next connection, so you can repeat the run without restarting it. Stop a run early with Ctrl+C.
Read the Result
The policy walks a grid of nine goals and logs every step:
A goal counts as reached once the fingertip stays within 5 cm for one second. A goal that
is not reached within 10 seconds logs a timed out line with the fingertip position, and
the policy moves on. Distance to goal lines appear every two seconds while the fingertip
is outside the tolerance.
Filter the log for the result:
Success criterion: the first Completed all 9 goals (9/9 achieved, 100.0%) line
appears before any timed out line.
📄 Example results
These results come from three headless runs of the default policy and configuration:
They were measured with Isaac Sim 6.2.0.1 on an NVIDIA RTX 5880 Ada Generation GPU shared with other workloads. Test date and commit were not recorded.
Goal timers use wall time, so a much slower host can time out goals that the policy would otherwise reach.
Check the Loop Timing
The simulator, the policy and the controller run on different clocks:
Each simulator frame advances physics by 10 ms, publishes the joint state and waits up to 10 ms for a command. The controller sends one command per state while it holds a fresh policy target. The policy runs every 20 ms of wall time on the freshest state. If the simulator runs slower than real time, the policy acts more often per physics step than it did in training.
When the run stops, the application prints a summary:
Read it line by line:
- Repeated physics steps should be zero. A repeated step means the simulator sent a frame without advancing physics. If it is not, see Troubleshooting.
- Skipped inferences happen when no joint state arrived in the last 20 ms. A few are normal. Many mean that simulator frames take longer than 20 ms.
- Expired policy targets happen when no new policy output arrived within 40 ms. The controller sends no command for those states and the arm holds its last target. If every target expires, frames take longer than 40 ms and the arm never leaves its start pose.
Isaac Sim also logs receive-timeout warnings in its Kit log; the Isaac Sim reference explains which ones are expected.
What the Simulation Does Not Show
The simulated arm uses the SO-101 system-identification asset, so its start pose, drives, speed and time base differ from the real arm. Good simulation results check the pipeline; only hardware shows actuator behavior such as how quickly a loaded servo closes a 0.05 rad error, overshoot, and settling under gravity. The servo speed and acceleration diagnostic measures step responses on the real arm. Validate motion on the real arm in Part 3.
❓ How does the simulated arm differ from the real one?
Troubleshooting
⚠️ Simulation problems
Summary
In this part you learned how to:
- Run a policy in Isaac Sim: build, configure an exported policy and run a bounded closed-loop session.
- Judge the run: read goal logs against a clear success criterion.
- Check the timing: read the run summary for repeated steps, skipped inferences and expired targets.
- Know the limits: which differences between the simulated and real arm need hardware.
Next Steps
- Part 3: Deployment: calibrate the arm and run the policy on hardware.
- Part 4: Benchmark: measure the pipeline’s latency and cadence.
- SO-101 reach reference: every
simmode option and timing detail.