The Isaac Platform#

NVIDIA Isaac is a robotics development platform that brings together simulation, learning, AI models, and reference workflows.

One part of NVIDIA Isaac is Isaac Lab, which is an open source, simulation framework for robot learning accelerated by graphics processing units (GPUs), designed to train robot policies at scale. It connects simulated scenes and sensors to the observations, actions, rewards, resets, and learning algorithms that define a task.

Isaac Lab does not directly implement physics, rendering, or reinforcement learning; instead, it provides a shared interface to those capabilities and contains several implementations for each interface. For example, the same task can run with the supported PhysX or Newton physics backends (or you can introduce a new physics backend for your problem). This module uses the Newton backend with the MJWarp solver, which is a good default combination.

A grid of fifteen Isaac Lab simulation scenes showing robot arms handling objects in varied rooms, mobile manipulators, and quadruped robots.

Isaac Lab provides a wide range of robot learning facilities, including common tasks, robots, and environments.#

Frameworks, Physics Engines, and Libraries#

The repositories below provide the learning, evaluation, simulation, and scene capabilities used with Isaac Lab. Their roles differ: some provide a backend, some build on Isaac Lab, and others supply supporting libraries.

Isaac Lab

Role: Robot Learning Tasks and Training Interfaces

Isaac Lab is a simulation framework for robot learning. It defines the observations, actions, rewards, resets, and environments for robot learning. It connects the task to physics and rendering backends and to learning libraries. This is where you configure the robot and task for the module.

Isaac Lab-Arena

Role: Task Composition and Policy Evaluation

Isaac Lab-Arena is an open source framework for policy setup and evaluation in simulation. It builds on Isaac Lab to compose evaluation environments from scenes, robots, and tasks. It runs policy evaluations and collects metrics to help you measure what a policy can do and where it fails.

Newton

Role: Extensible Physics Simulation

Newton is a GPU-accelerated physics engine built on Warp. It supplies the physics backend used by Isaac Lab in this module. For custom simulation needs, start with Newton’s existing solvers or develop a custom solver within Newton. Check that the Isaac Lab integration supports the features your task needs.

MuJoCo Warp

Role: GPU-Accelerated Solver Used Through Newton

MuJoCo Warp (MJWarp) implements MuJoCo simulation primitives using Warp for parallel GPU execution. Newton integrates it as a solver option. The module’s newton_mjwarp preset selects Newton with this solver to advance the simulated robot and its environment.

Warp

Role: Accelerated Numerical Computation

Warp is a Python framework for accelerated simulation. It compiles Python kernel functions into code that runs on CPUs or NVIDIA GPUs. Newton and MuJoCo Warp use it for their physics calculations. In Isaac Lab, you normally configure tasks and robots; you work with Warp directly when you need custom computations beneath that layer.

PhysX

Role: An Alternative Physics Engine

PhysX is NVIDIA’s physics SDK. It powers Isaac Sim’s established physics workflow and is also available through Isaac Lab’s PhysX backends. It provides another physics option for an Isaac Lab task, alongside the Newton backend used in this module.

Isaac Sim

Role: Robotics Simulation and Integration

Isaac Sim is an open source reference framework built on Omniverse libraries for robotics simulation. It brings together scene authoring, robot import, physics, RTX rendering, simulated sensors, and ROS 2 integration. Isaac Lab can use its simulation capabilities, and can also run a standalone Newton workflow without installing Isaac Sim.

OpenUSD

Role: Scene and Asset Description

OpenUSD provides a shared way to describe and compose scenes, including robots, geometry, materials, and physics properties. It lets simulation frameworks exchange scene assets used in Isaac Lab workflows. It describes the scene; the selected physics backend advances the simulation.

How the Pieces Fit Together#

Isaac Lab can use Isaac Sim’s simulation capabilities. It can also use standalone backends: the Isaac Lab 3 architecture supports running Newton without installing Isaac Sim, often called Kit-less operation (Kit is the application framework that Isaac Sim is built on).

Isaac Lab sits above two alternative physics stacks. One uses Isaac Sim with PhysX; the other uses Newton with the MJWarp solver and can run without Isaac Sim.

Two of the physics stacks Isaac Lab supports: Isaac Sim with PhysX, and Newton with MJWarp.#

Omniverse supplies libraries and development tools for simulation and physical AI applications. Isaac Sim packages these capabilities into a robotics application. Isaac Lab can also use standalone libraries such as ovphysx for physics and ovrtx for rendering and sensor simulation through its backend integrations.

The Isaac Lab framework is split into packages with different responsibilities:

  • isaaclab provides the core scene, asset, sensor, environment, manager, actuator, controller, and terrain interfaces.

  • isaaclab_newton, isaaclab_physx, and isaaclab_ov provide backend physics implementations.

  • isaaclab_assets contains ready-to-configure robots and sensors.

  • isaaclab_tasks contains registered example environments.

  • isaaclab_rl adapts environments for reinforcement learning (RL) libraries such as RSL-RL, skrl, Stable Baselines 3, and RL Games.

You may also encounter Isaac Gym in older papers and tutorials. It is deprecated; use Isaac Lab for new robot learning projects. The older IsaacGymEnvs, OmniIsaacGymEnvs, and Orbit task collections have also been superseded by Isaac Lab.

Follow a Pick-And-Place Workflow#

For example, you might prepare a robot and laboratory in Isaac Sim, train a pick-and-place policy in Isaac Lab using Newton, and evaluate variations of the task with Isaac Lab-Arena. You can then connect the policy to the broader robot software stack in Isaac Sim before testing it on hardware.

Throughout that workflow, OpenUSD describes scene assets, Warp accelerates Newton’s computations, and Omniverse supplies reusable simulation and rendering capabilities. Moving between configurations still requires checking robot parameters, sensor settings, and the policy’s observation and action interfaces.

Review#

Check Your Understanding#

What role does Isaac Lab play in a robot learning workflow?

Answer

Isaac Lab defines robot learning tasks and connects them to physics backends and learning libraries. The selected physics backend advances the simulation; the learning library updates the policy.

Apply What You Learned#

How would you explain the roles of Isaac Lab, Newton, MuJoCo Warp, and a learning library in the cartpole workflow? Trace how the task definition, physics calculations, and policy updates fit together.

Agent Tutor Prompt

Copy this prompt and include your answer:

Review my answer to this question: How would you explain the roles of Isaac Lab, Newton, MuJoCo Warp, and a learning library in the cartpole workflow? Trace how the task definition, physics calculations, and policy updates fit together.

My answer: [paste your answer here].

Check my reasoning about task definition, the distinction between a physics engine and its solver, and the role of the learning library. Ask for missing context rather than assuming it. Identify what I explained well, point out one gap or misconception, and ask a follow-up question that helps me improve my answer. Let me revise it before supplying a complete answer.

Key Takeaways#

  • Isaac Lab defines robot learning tasks and connects them to learning libraries.

  • Newton advances physics; MuJoCo Warp supplies the solver used in this module.

  • Physics, policy observations, rendering, and evaluation have distinct roles.

Resources#

In addition to robot learning and simulation, NVIDIA Isaac provides other robotics capabilities:

  • Isaac ROS provides accelerated ROS 2 software, AI models, and deployment tools for building robots.

  • Isaac Manipulator provides Isaac ROS packages and reference workflows for robotic arms, including perception and motion planning.

  • Isaac GR00T is an open platform for developing humanoid robots with foundation models, data pipelines, and simulation workflows.

  • NVIDIA Cosmos provides world foundation models and tools for generating and processing data for physical AI.