# The Isaac Platform [NVIDIA Isaac](https://developer.nvidia.com/isaac) is a robotics development platform that brings together simulation, learning, AI models, and reference workflows. One part of NVIDIA Isaac is **Isaac Lab**, which is an open source, **simulation framework for robot learning** accelerated by graphics processing units (GPUs), designed to train robot policies at scale. It connects simulated scenes and sensors to the observations, actions, rewards, resets, and learning algorithms that define a task. Isaac Lab does **not** directly implement physics, rendering, or reinforcement learning; instead, it provides a shared interface to those capabilities and contains several implementations for each interface. For example, the same task can run with the supported PhysX or [Newton](https://newton-physics.github.io/newton/) physics backends (or you can introduce a new physics backend for your problem). This module uses the Newton backend with the MJWarp solver, which is a good default combination. ![A grid of fifteen Isaac Lab simulation scenes showing robot arms handling objects in varied rooms, mobile manipulators, and quadruped robots.](images/robot-learning-isaac-lab.jpg) ## Frameworks, Physics Engines, and Libraries The repositories below provide the learning, evaluation, simulation, and scene capabilities used with Isaac Lab. Their roles differ: some provide a backend, some build on Isaac Lab, and others supply supporting libraries. **Role: Robot Learning Tasks and Training Interfaces** [Isaac Lab](https://github.com/isaac-sim/IsaacLab) is a simulation framework for robot learning. It defines the observations, actions, rewards, resets, and environments for robot learning. It connects the task to physics and rendering backends and to learning libraries. This is where you configure the robot and task for the module. **Role: Task Composition and Policy Evaluation** [Isaac Lab-Arena](https://github.com/isaac-sim/IsaacLab-Arena) is an open source framework for policy setup and evaluation in simulation. It builds on Isaac Lab to compose evaluation environments from scenes, robots, and tasks. It runs policy evaluations and collects metrics to help you measure what a policy can do and where it fails. **Role: Extensible Physics Simulation** [Newton](https://github.com/newton-physics/newton) is a GPU-accelerated physics engine built on Warp. It supplies the physics backend used by Isaac Lab in this module. For custom simulation needs, start with Newton’s existing solvers or [develop a custom solver](https://newton-physics.github.io/newton/latest/api/_generated/newton.solvers.SolverBase.html) within Newton. Check that the Isaac Lab integration supports the features your task needs. **Role: GPU-Accelerated Solver Used Through Newton** [MuJoCo Warp (MJWarp)](https://github.com/google-deepmind/mujoco_warp) implements MuJoCo simulation primitives using Warp for parallel GPU execution. Newton integrates it as a solver option. The module’s `newton_mjwarp` preset selects Newton with this solver to advance the simulated robot and its environment. **Role: Accelerated Numerical Computation** [Warp](https://github.com/NVIDIA/warp) is a Python framework for accelerated simulation. It compiles Python kernel functions into code that runs on CPUs or NVIDIA GPUs. Newton and MuJoCo Warp use it for their physics calculations. In Isaac Lab, you normally configure tasks and robots; you work with Warp directly when you need custom computations beneath that layer. **Role: An Alternative Physics Engine** [PhysX](https://github.com/NVIDIA-Omniverse/PhysX) is NVIDIA’s physics SDK. It powers Isaac Sim’s established physics workflow and is also available through Isaac Lab’s PhysX backends. It provides another physics option for an Isaac Lab task, alongside the Newton backend used in this module. **Role: Robotics Simulation and Integration** [Isaac Sim](https://github.com/isaac-sim/IsaacSim) is an open source reference framework built on Omniverse libraries for robotics simulation. It brings together scene authoring, robot import, physics, RTX rendering, simulated sensors, and ROS 2 integration. Isaac Lab can use its simulation capabilities, and can also run a standalone Newton workflow without installing Isaac Sim. **Role: Scene and Asset Description** [OpenUSD](https://github.com/PixarAnimationStudios/OpenUSD) provides a shared way to describe and compose scenes, including robots, geometry, materials, and physics properties. It lets simulation frameworks exchange scene assets used in Isaac Lab workflows. It describes the scene; the selected physics backend advances the simulation. ## How the Pieces Fit Together Isaac Lab can use Isaac Sim’s simulation capabilities. It can also use standalone backends: the [Isaac Lab 3 architecture](https://isaac-sim.github.io/IsaacLab/release/3.0.0/source/setup/ecosystem.html) supports running Newton without installing Isaac Sim, often called *Kit-less* operation (Kit is the application framework that Isaac Sim is built on). ![Isaac Lab sits above two alternative physics stacks. One uses Isaac Sim with PhysX; the other uses Newton with the MJWarp solver and can run without Isaac Sim.](images/isaac-lab-physics-stacks.svg) [Omniverse](https://developer.nvidia.com/omniverse) supplies libraries and development tools for simulation and physical AI applications. Isaac Sim packages these capabilities into a robotics application. Isaac Lab can also use standalone libraries such as **ovphysx** for physics and **ovrtx** for rendering and sensor simulation through its backend integrations. The Isaac Lab framework is split into packages with different responsibilities: - **`isaaclab`** provides the core scene, asset, sensor, environment, manager, actuator, controller, and terrain interfaces. - **`isaaclab_newton`, `isaaclab_physx`, and `isaaclab_ov`** provide backend physics implementations. - **`isaaclab_assets`** contains ready-to-configure robots and sensors. - **`isaaclab_tasks`** contains registered example environments. - **`isaaclab_rl`** adapts environments for reinforcement learning (RL) libraries such as RSL-RL, skrl, Stable Baselines 3, and RL Games. You may also encounter [Isaac Gym](https://developer.nvidia.com/isaac-gym) in older papers and tutorials. It is **deprecated**; use Isaac Lab for new robot learning projects. The older IsaacGymEnvs, OmniIsaacGymEnvs, and Orbit task collections have also been superseded by Isaac Lab. ## Follow a Pick-And-Place Workflow For example, you might prepare a robot and laboratory in **Isaac Sim**, train a pick-and-place policy in **Isaac Lab** using **Newton**, and evaluate variations of the task with **Isaac Lab-Arena**. You can then connect the policy to the broader robot software stack in Isaac Sim before testing it on hardware. Throughout that workflow, **OpenUSD** describes scene assets, **Warp** accelerates Newton’s computations, and **Omniverse** supplies reusable simulation and rendering capabilities. Moving between configurations still requires checking robot parameters, sensor settings, and the policy’s observation and action interfaces. ## Review ### Check Your Understanding What role does Isaac Lab play in a robot learning workflow? ### Answer Isaac Lab defines robot learning tasks and connects them to physics backends and learning libraries. The selected physics backend advances the simulation; the learning library updates the policy. ### Apply What You Learned How would you explain the roles of Isaac Lab, Newton, MuJoCo Warp, and a learning library in the cartpole workflow? Trace how the task definition, physics calculations, and policy updates fit together. ### Agent Tutor Prompt Copy this prompt and include your answer: > Review my answer to this question: How would you explain the roles of Isaac Lab, Newton, MuJoCo Warp, and a learning library in the cartpole workflow? Trace how the task definition, physics calculations, and policy updates fit together. > My answer: [paste your answer here]. > Check my reasoning about task definition, the distinction between a physics engine and its solver, and the role of the learning library. Ask for missing context rather than assuming it. Identify what I explained well, point out one gap or misconception, and ask a follow-up question that helps me improve my answer. Let me revise it before supplying a complete answer. ### Key Takeaways - Isaac Lab defines robot learning tasks and connects them to learning libraries. - Newton advances physics; MuJoCo Warp supplies the solver used in this module. - Physics, policy observations, rendering, and evaluation have distinct roles. ### Resources In addition to robot learning and simulation, NVIDIA Isaac provides other robotics capabilities: - [Isaac ROS](https://developer.nvidia.com/isaac/ros) provides accelerated ROS 2 software, AI models, and deployment tools for building robots. - [Isaac Manipulator](https://nvidia-isaac-ros.github.io/v/release-4.1/reference_workflows/isaac_for_manipulation/index.html) provides Isaac ROS packages and reference workflows for robotic arms, including perception and motion planning. - [Isaac GR00T](https://developer.nvidia.com/isaac/gr00t) is an open platform for developing humanoid robots with foundation models, data pipelines, and simulation workflows. - [NVIDIA Cosmos](https://developer.nvidia.com/cosmos) provides world foundation models and tools for generating and processing data for physical AI.