How-to Guides | Isaac Lab

Find step-by-step guides for building with Isaac Lab, grouped by topic.

Simulation Fundamentals

Creating an empty scene — Launch an empty simulation and learn the core startup sequence.
Spawning prims into the scene — Add lights, ground planes, and primitive shapes to a simulation stage.
Deep-dive into launching the simulation — Configure and launch simulation applications from Python and the command line.

Interacting with a rigid object — Create, reset, and command a cone; includes the runnable run_rigid_object.py example.
Interacting with an articulation — Work with joint state, commands, and articulation data.
Interacting with a deformable object — Spawn and manipulate deformable bodies in a scene.
Interacting with a surface gripper — Attach and release rigid objects with a surface gripper.
Importing a new asset — Convert URDF, MJCF, or mesh assets into USD for use in Isaac Lab.
Robot and articulation configuration — Reuse or author a robot configuration, with a Jetbot/Dofbot worked example.
Making a physics prim fixed — Convert a floating asset into a fixed object in the simulation.
Spawning multiple assets — Batch rigid objects and vary asset configurations across environments.
Managing asset downloads — Cache remote assets, migrate Nucleus setups, and select a regional asset service.
Preparing an asset for Newton with MJWarp — Prepare an asset and task to run with the Newton MJWarp physics preset.
Exploring MPM materials and coupling — Compare material response, fluid surface reconstruction, and robot—particle coupling.

Using the interactive scene — Compose assets and sensors with the higher-level interactive scene interface.
Cloning environments — Choose cloning strategies, build heterogeneous scenes, and filter collisions.

Creating a manager-based base environment — Build a non-RL environment from reusable manager terms.
Creating a manager-based RL environment — Add rewards, terminations, curricula, and commands for reinforcement learning.
Creating a direct workflow RL environment — Implement an RL task with direct control over the environment loop.
Registering an environment — Register an Isaac Lab task with Gymnasium and expose its configurations.
Training with an RL agent — Launch training and inference with a supported reinforcement learning library.
Configuring an RL agent — Customize agent settings and training hyperparameters.
Modifying an existing direct RL environment — Extend and adjust a direct workflow task without rebuilding it from scratch.
Policy inference in a USD environment — Run a trained policy against an environment defined in a USD stage.
Wrapping environments — Adapt Isaac Lab environments to external reinforcement learning interfaces.
Adding your own learning library — Integrate an additional learning framework with Isaac Lab tasks.
Running scripted state machines — Drive environments with deterministic state-machine policies.
Curriculum utilities — Change environment parameters dynamically during training.
Transferring policies between PhysX and Newton — Validate and evaluate policy checkpoints across the PhysX and Newton backends.

Adding sensors on a robot — Add camera, ray-caster, and contact sensors to an environment.
Saving rendered images and 3D re-projection — Save camera outputs and reconstruct point clouds from depth images.
Select and configure a Renderer — Choose Newton Warp, OVRTX, or legacy Isaac RTX and tune the selected renderer.
Streaming a camera view in a visualizer — Show tiled ground-truth camera feeds from many environments in a live visualizer panel.
Capturing sensor frames during training — Record selected sensor outputs from a running training job.
Recording video — Record mp4 clips from a visualizer or scene sensor, from the CLI or an environment config.
Creating visualization markers — Draw frames, arrows, and custom shapes over the scene to debug per-environment state.

Using a task-space controller — Control a robot end effector with differential inverse kinematics.
Using an operational space controller — Apply operational-space control to a robot manipulator.

Reproducibility and determinism — Seed runs and request deterministic physics, rendering, and training.
Working with ProxyArray — Consume asset and sensor data through Torch and Warp while keeping retained views valid.
Simulation performance — Diagnose bottlenecks and improve simulation throughput.
Tuning the PhysX solver — Tune timestep, iterations, and contacts to diagnose PhysX solver stability.
Tuning the MJWarp solver — Tune Newton MuJoCo-Warp contact capacity, timestep, friction, and convergence.
Tuning the Kamino solver — Tune Newton Kamino PADMM or DVI convergence, stabilization, timestep, and substeps.
Tuning the VBD solver — Tune Newton Vertex Block Descent (VBD) materials, contacts, cloth, soft bodies, and proxy or ADMM coupling.
Tuning the MPM solver — Tune Newton Material Point Method (MPM) resolution, timestep, convergence, and material parameters.

Setting up Isaac Capture with CloudXR — Connect XR devices through CloudXR for immersive teleoperation.
Setting up Haply teleoperation — Use Haply devices for robot control with directional force feedback.

Recording animations of simulations — Capture simulation state and export an animation.
Mastering Omniverse for robotics — Find Omniverse workflows and resources relevant to Isaac Lab.
Profiling Isaac Lab with Nsight Systems — Capture and inspect runtime traces with NVIDIA Nsight Systems.