Scene and Simulation Setup#

In this lesson, we’ll bootstrap Isaac Lab, load the Franka cube task scene, and configure Newton as the physics engine. By the end you’ll have a live environment stepping in simulation with the robot holding its default pose.

In this lesson, we will:

Bootstrap an interactive notebook with AppLauncher and relate it to the scripts’ launch_simulation lifecycle.
Inspect how the task scene assembles a Franka, a cube, a table, and a ground plane.
Configure the three-layer Newton simulation stack.
Build the environment and step it with zero actions.

Part 1: Initial Setup and Imports#

For an interactive notebook, AppLauncher starts the application and enables the Viser web visualizer before the environment is constructed. The canonical external-project scripts use Isaac Lab 3.0’s launch_simulation context manager instead; this lets them resolve the task configuration and selected backends before starting the runtime.

from isaaclab.app import AppLauncher

app_launcher = AppLauncher({"visualizer": ["viser"], "visualizer_max_worlds": 4})
simulation_app = app_launcher.app

import gymnasium as gym
import torch

import isaaclab_tasks  # noqa: F401
import franka_cube.tasks  # noqa: F401

from franka_cube.tasks.direct.franka_cube.franka_cube_env import FrankaCubeEnv
from franka_cube.tasks.direct.franka_cube.franka_cube_env_cfg import FrankaCubeEnvCfg

Important

The runtime must be active before constructing or stepping the environment. In the notebook, create AppLauncher before the environment. In random_agent.py, train.py, and play.py, keep environment creation inside with launch_simulation(env_cfg, args_cli):.

The notebook version also defines display helpers (for example display_active_viewer) that embed the live Viser viewer inside Jupyter. Those are display conveniences and are not required to run the environment.

Part 2: Defining the Scene#

The FrankaCubeEnv already implements the scene used by the task: a fixed-base Franka, a rigid cube, a SeattleLab table, and a shifted ground plane below the table. The scene is assembled in _setup_scene, which registers the Newton contact callback, spawns the robot and cube, adds the table and ground, clones the per-environment copies, and adds a dome light.

import isaaclab.sim as sim_utils
from isaaclab.assets import Articulation, RigidObject
from isaaclab.sim.spawners.from_files import GroundPlaneCfg, spawn_ground_plane
from isaaclab.utils.assets import ISAAC_NUCLEUS_DIR


def _setup_scene(self):
    self._register_newton_contact_callback()
    self.robot = Articulation(self.cfg.robot_cfg)
    self.cube = RigidObject(self.cfg.cube)
    spawn_ground_plane(prim_path="/World/ground", cfg=GroundPlaneCfg(), translation=(0.0, 0.0, -1.05))
    table_cfg = sim_utils.UsdFileCfg(usd_path=f"{ISAAC_NUCLEUS_DIR}/Props/Mounts/SeattleLabTable/table_instanceable.usd")
    table_cfg.func(
        "/World/envs/env_.*/Table",
        table_cfg,
        translation=(0.5, 0.0, 0.0),
        orientation=(0.0, 0.0, 0.70711, 0.70711),
    )
    self.scene.clone_environments(copy_from_source=False)
    if self.device == "cpu":
        self.scene.filter_collisions(global_prim_paths=[])
    self.scene.articulations["robot"] = self.robot
    self.scene.rigid_objects["cube"] = self.cube
    light_cfg = sim_utils.DomeLightCfg(intensity=2000.0, color=(0.75, 0.75, 0.75))
    light_cfg.func("/World/Light", light_cfg)

The call to clone_environments is what makes Isaac Lab fast: it replicates the single scene into many parallel environments on the GPU, which is essential for RL where we train across hundreds or thousands of environments at once.

Part 3: Setting Up the Simulation#

The key to running Isaac Lab with Newton is a three-layer physics configuration: MJWarpSolverCfg (the solver itself), wrapped by NewtonCfg (the physics engine), wrapped by SimulationCfg (top-level simulation parameters).

Parameter	Value	Description
`solver`	`"newton"`	Uses the Newton contact solver
`integrator`	`"implicitfast"`	Integration method (`"euler"`, `"rk4"`, `"implicit"`, `"implicitfast"`)
`njmax`	`2000`	Maximum number of constraints per world
`nconmax`	`1000`	Maximum number of contact points per world
`impratio`	`100.0`	Frictional-to-normal constraint impedance ratio
`cone`	`"elliptic"`	Contact friction cone (`"pyramidal"` or `"elliptic"`)
`iterations`	`20`	Number of solver iterations
`ls_iterations`	`100`	Number of line-search iterations
`ccd_iterations`	`80`	Extra continuous-collision passes for difficult contacts
`ls_parallel`	`True`	Enable parallel line search in MuJoCo
`use_mujoco_contacts`	`False`	Keep Newton’s global contact defaults in control

import isaaclab.sim as sim_utils
from isaaclab_newton.physics import MJWarpSolverCfg, NewtonCfg
from isaaclab.sim import SimulationCfg

notebook_env_cfg.decimation = 2

notebook_env_cfg.solver_cfg = MJWarpSolverCfg(
    solver="newton",
    integrator="implicitfast",
    njmax=2000,
    nconmax=1000,
    impratio=100.0,
    cone="elliptic",
    update_data_interval=2,
    iterations=20,
    ls_iterations=100,
    ccd_iterations=80,
    ls_parallel=True,
    use_mujoco_contacts=False,
)

notebook_env_cfg.newton_cfg = NewtonCfg(
    solver_cfg=notebook_env_cfg.solver_cfg,
    num_substeps=5,
    debug_mode=False,
)

notebook_env_cfg.sim = SimulationCfg(
    dt=1 / 120,
    render_interval=notebook_env_cfg.decimation,
    physics=notebook_env_cfg.newton_cfg,
    physics_material=sim_utils.RigidBodyMaterialCfg(
        friction_combine_mode="multiply",
        restitution_combine_mode="multiply",
        static_friction=1.0,
        dynamic_friction=1.0,
        restitution=0.0,
    ),
)

Notice how the same solver ideas from Newton Fundamentals (integrator, iterations, cone type, contact limits) reappear here, just wrapped in Isaac Lab’s configuration objects.

Quick Test: Step the Environment#

With the configuration in place, we build the environment through the gym API and run it with zero actions so the robot holds its default pose. Building the environment takes a few minutes the first time while kernels compile.

env_cfg = FrankaCubeEnvCfg()
env_cfg.scene.num_envs = 32
env = gym.make("Template-Franka-Cube-Direct-v0", cfg=env_cfg)
obs, info = env.reset()

# Run with zero actions so the robot holds its default pose
for i in range(1000):
    with torch.inference_mode():
        actions = torch.zeros(
            (env.unwrapped.num_envs, env.unwrapped.cfg.action_space),
            device=env.unwrapped.device,
        )
        obs, reward, terminated, truncated, info = env.step(actions)

print("Done! The scene is at rest with the robot in its default pose.")

Running It End to End#

The canonical, runnable version of everything above lives in the franka_cube extension. Its scripts use launch_simulation to own startup and shutdown. To watch the untrained scene step with a random agent, run this from a JupyterLab terminal:

python scripts/random_agent.py \
  --task Template-Franka-Cube-Direct-v0 --num_envs 32 --viz kit

For a bounded headless smoke test, replace --viz kit with --viz none --max_steps 200.

Key Takeaways#

You bootstrapped the notebook with AppLauncher, related that flow to the scripts’ launch_simulation context, saw how the Franka cube scene is assembled and cloned, and configured Newton through the three-layer MJWarpSolverCfg → NewtonCfg → SimulationCfg stack. With a live environment stepping, we’re ready to define what the agent actually senses, does, and is rewarded for.