# Simulation to Reality In this lesson, we’ll provide an overview of the simulation-to-reality gap. When you’re ready to explore transfer methods in more depth, continue to [Transfer Policies From Simulation to Reality](https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-lab/2026-10.0/transfer-policies-from-simulation-to-reality/index.md). ![Varied simulated workcells produce a trained policy, which is transferred and tested on physical hardware.](images/robot-learning-sim2real-simplified.svg) Simulation gives you repeated trials, controlled resets, and thousands of parallel environments. It is also an *approximation*. Differences in friction, contact, actuator response, cameras, lighting, and timing can make a policy that works in simulation fail on hardware. This mismatch is the **simulation-to-reality gap**, also called the **reality gap**. **Sim-to-real** and **sim2real** are interchangeable names for transferring a policy from simulation to a physical robot. Successful transfer remains a difficult, task-dependent problem. It depends on the robot, sensors, control timing, contact behavior, training conditions, and the performance required. A successful demonstration on one setup does not establish that the same policy will transfer to another. Where reality gap describes differences between the simulated and real systems, the **performance gap** describes how much a particular policy’s measured task performance changes between them. A simulator need not reproduce every detail of the real world for a policy to transfer successfully. You can think of simulation as the source domain and the physical laboratory as the target domain. **Transfer** succeeds when the observations and dynamics seen on the robot remain close enough to the policy’s training experience for it to choose useful actions. ## Locate the Gap Before you tackle the reality gap, ask what might differ between simulation and hardware. These categories can help you work through that question: - **Dynamics:** mass, inertia, friction, compliance, and contact change how an action moves the robot. - **Observations:** camera calibration, lighting, texture, sensor noise, latency, and missing measurements change what the policy perceives. - **Actuation and control:** motor strength, backlash, command filtering, and control delay change how policy actions become physical motion. - **Task setup:** object pose, geometry, fixtures, and reset accuracy change the situations the policy encounters. - **Software and interfaces:** unit conventions, joint ordering, action scaling, control frequency, and preprocessing can create failures even when the physics model is good. The last category is especially important because randomization cannot repair an incorrect interface. Verify tensor ordering, units, frame conventions, clipping, and timing before treating a failure as a learning problem. One way to do so is to run a simple scripted (not learned) policy in both simulation and on hardware, then compare the resulting observations and actions. If they differ, the problem is likely in the interface rather than the learned policy. ## Closing the Reality Gap [Aljalbout et al., *The Reality Gap in Robotics*](https://arxiv.org/abs/2510.20808) distinguish two complementary approaches: - **Reducing the gap:** improve the simulation or system design so the simulated and physical systems behave more similarly. - **Overcoming the gap:** train or adapt a policy to perform well despite the differences that remain. For an Isaac Lab task, calibrating actuator parameters helps reduce the gap, and training across plausible parameter variations helps a policy overcome the remaining mismatch. Successful transfer does not require a perfect simulator, but it does require evaluating the policy on the target hardware and updating the simulation or policy based on those results. No single technique solves every transfer problem. Choose methods that address a measured or plausible source of uncertainty: - **System identification** uses measurements from the real system to estimate parameters such as mass, friction, latency, and actuator response. You then use these estimates to improve the simulation model. - **Domain randomization** samples conditions around that model during training. Randomize plausible ranges, not arbitrary extremes. If the policy cannot learn under the final range, introduce it gradually with a curriculum (a schedule that widens the ranges as training progresses). - **Observation augmentation** changes images or sensor values in ways that resemble real variation, such as brightness changes, cropping, noise, or delay. - **Robust policy design** removes unnecessary privileged inputs, includes useful observation history, and discourages violent or highly sensitive actions. - **Adaptation or fine-tuning** uses real data to adjust a representation or policy. This can improve transfer, but real interaction is slower, costlier, and potentially unsafe. Domain randomization (DR) does not make the simulator “more real” in every episode. Instead, it teaches the policy not to depend on one exact simulated world. For example, if you sample reasonable friction and delay values, a policy that works across that range is less likely to depend on the simulator’s single default value. ![Plausible variations in lighting, friction, and control delay feed training of one policy across varied simulation conditions. The policy is then tested on hardware for task success and failure types.](images/domain-randomization.svg) To assess transfer, compare the same policy against the same task-level measure in simulation and on hardware, such as placement success rate. Record the test conditions and failure types in both settings. A small gap between the two is useful evidence, but the hardware success rate still matters: equally poor performance in both settings is not successful transfer. ## Review ### Check Your Understanding Does success in simulation establish success on the real robot? ### Answer No. Simulation results establish performance only under the simulated conditions tested. Real-world measurements and controlled hardware evaluation are still needed. ### Apply What You Learned For one suspected simulation-to-reality mismatch, what experiment would you design and what could its results establish? Specify one changed factor, a measured outcome, and what would still require hardware evidence. ### Agent Tutor Prompt Copy this prompt and include your answer: > Review my answer to this question: For one suspected simulation-to-reality mismatch, what experiment would you design and what could its results establish? Specify one changed factor, a measured outcome, and what would still require hardware evidence. > My answer: [paste your answer here]. > Check my reasoning about one changed factor, measurable outcome, plausible mismatch, and limits of simulation evidence. Ask for missing context rather than assuming it. Identify what I explained well, point out one gap or misconception, and ask a follow-up question that helps me improve my answer. Let me revise it before supplying a complete answer. ### Key Takeaways - Sim-to-real and sim2real describe the same transfer process. - Measure likely mismatches before choosing randomization or adaptation. - Isaac Lab experiments can test hypotheses; hardware measurements are needed to establish transfer. ### Resources - [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://arxiv.org/abs/2510.20808) distinguishes reducing the reality gap from overcoming it (Section 4), and separates evaluation of system differences from evaluation of policy transfer (Section 5). - [Isaac Lab real-robot reach-policy example](https://isaac-sim.github.io/IsaacLab/release/3.0.0/source/policy_deployment/04_reach/reach_policy.html) shows actuator, friction, and reset randomization in a deployment workflow. - [Isaac Lab policy transfer between PhysX and Newton](https://isaac-sim.github.io/IsaacLab/release/3.0.0/source/how-to/transfer_policies_between_physx_and_newton.html) explains how to choose physically plausible randomization ranges and when to use a curriculum. - [Sim-to-Real Transfer of Robotic Control With Dynamics Randomization](https://arxiv.org/abs/1710.06537) is an early primary research example of training a manipulation policy across randomized dynamics and deploying it on a real robot.