> This page is for version Release 3.0 (default).
> For other versions, use one of these documentation indexes:
> - Release 3.0 (default): https://docs.nvidia.com/isaaclab/3.0/llms.txt
> - Release 2.3: https://docs.nvidia.com/isaaclab/2.3/llms.txt
> - Release 2.2: https://docs.nvidia.com/isaaclab/2.2/llms.txt
> - Release 2.1: https://docs.nvidia.com/isaaclab/2.1/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/isaaclab/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/isaaclab/_mcp/server.

# Adding sensors on a robot

While the asset classes allow us to create and simulate the physical embodiment of the robot,
sensors help in obtaining information about the environment. They typically update at a lower
frequency than the simulation and are useful for obtaining different proprioceptive and
exteroceptive information. For example, a camera sensor can be used to obtain the visual
information of the environment, and a contact sensor can be used to obtain the contact
information of the robot with the environment.

In this tutorial, we will see how to add different sensors to a robot. We will use the
ANYmal-C robot for this tutorial. The ANYmal-C robot is a quadrupedal robot with 12 degrees
of freedom. It has 4 legs, each with 3 degrees of freedom. The robot has the following
sensors:

* A camera sensor on the head of the robot which provides RGB-D images
* A height scanner sensor that provides terrain height information
* Contact sensors on the feet of the robot that provide contact information

We continue this tutorial from the previous tutorial on [Using the Interactive Scene](/source/how-to/create_scene),
where we learned about the `scene.InteractiveScene` class.

## The Code

The tutorial corresponds to the `add_sensors_on_robot.py` script in the
`scripts/tutorials/04_sensors` directory.

```python
# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause

"""
This script demonstrates how to add and simulate on-board sensors for a robot.

We add the following sensors on the quadruped robot, ANYmal-C (ANYbotics):

* USD-Camera: This is a camera sensor that is attached to the robot's base.
* Height Scanner: This is a height scanner sensor that is attached to the robot's base.
* Contact Sensor: This is a contact sensor that is attached to the robot's feet.

~~~bash
# Usage
uv run python scripts/tutorials/04_sensors/add_sensors_on_robot.py --viz kit
~~~

"""

"""Parse the command-line arguments first."""

import argparse
from typing import TYPE_CHECKING

from isaaclab.app import add_launcher_args, launch_simulation

# add argparse arguments
parser = argparse.ArgumentParser(description="Tutorial on adding sensors on a robot.")
parser.add_argument("--num_envs", type=int, default=2, help="Number of environments to spawn.")
# append simulation launcher cli args
add_launcher_args(parser)
# parse the arguments
args_cli = parser.parse_args()
# The scene has a camera sensor, which requires the rendering extensions in headless launches.
args_cli.enable_cameras = True

"""Rest everything follows."""

import torch

import isaaclab.sim as sim_utils
from isaaclab.assets import ArticulationCfg, AssetBaseCfg
from isaaclab.scene import InteractiveSceneCfg
from isaaclab.sensors import CameraCfg, ContactSensorCfg, RayCasterCfg, patterns
from isaaclab.utils import configclass, instantiate, replace

if TYPE_CHECKING:
    from isaaclab.scene import InteractiveScene

##
# Pre-defined configs
##
from isaaclab_assets.robots.anymal import ANYMAL_C_CFG  # isort: skip

@configclass
class SensorsSceneCfg(InteractiveSceneCfg):
    """Design the scene with sensors on the robot."""

    # ground plane
    ground = AssetBaseCfg(prim_path="/World/defaultGroundPlane", spawn=sim_utils.GroundPlaneCfg())

    # lights
    dome_light = AssetBaseCfg(
        prim_path="/World/Light", spawn=sim_utils.DomeLightCfg(intensity=3000.0, color=(0.75, 0.75, 0.75))
    )

    # robot
    robot: ArticulationCfg = replace(ANYMAL_C_CFG, prim_path="{ENV_REGEX_NS}/Robot")

    # sensors
    camera = CameraCfg(
        prim_path="{ENV_REGEX_NS}/Robot/base/front_cam",
        update_period=0.1,
        height=480,
        width=640,
        data_types=["rgb", "distance_to_image_plane"],
        spawn=sim_utils.PinholeCameraCfg(
            focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
        ),
        offset=CameraCfg.OffsetCfg(pos=(0.510, 0.0, 0.015), rot=(0.5, -0.5, 0.5, -0.5), convention="ros"),
    )
    height_scanner = RayCasterCfg(
        prim_path="{ENV_REGEX_NS}/Robot/base",
        update_period=0.02,
        offset=RayCasterCfg.OffsetCfg(pos=(0.0, 0.0, 20.0)),
        ray_alignment="yaw",
        pattern_cfg=patterns.GridPatternCfg(resolution=0.1, size=[1.6, 1.0]),
        debug_vis=True,
        mesh_prim_paths=["/World/defaultGroundPlane"],
    )
    contact_forces = ContactSensorCfg(
        prim_path="{ENV_REGEX_NS}/Robot/.*_FOOT", update_period=0.0, history_length=6, debug_vis=True
    )

def run_simulator(sim: sim_utils.SimulationContext, scene: "InteractiveScene"):
    """Run the simulator."""
    # Define simulation stepping
    sim_dt = sim.get_physics_dt()
    sim_time = 0.0
    count = 0

    # Simulate physics
    while sim.is_running():
        # Reset
        if count % 500 == 0:
            # reset counter
            count = 0
            # reset the scene entities
            # root state
            # we offset the root state by the origin since the states are written in simulation world frame
            # if this is not done, then the robots will be spawned at the (0, 0, 0) of the simulation world
            root_pose = scene["robot"].data.default_root_pose.torch.clone()
            root_pose[:, :3] += scene.env_origins
            scene["robot"].write_root_pose_to_sim_index(root_pose=root_pose)
            root_vel = scene["robot"].data.default_root_vel.torch.clone()
            scene["robot"].write_root_velocity_to_sim_index(root_velocity=root_vel)
            # set joint positions with some noise
            joint_pos, joint_vel = (
                scene["robot"].data.default_joint_pos.torch.clone(),
                scene["robot"].data.default_joint_vel.torch.clone(),
            )
            joint_pos += torch.rand_like(joint_pos) * 0.1
            scene["robot"].write_joint_position_to_sim_index(position=joint_pos)
            scene["robot"].write_joint_velocity_to_sim_index(velocity=joint_vel)
            # clear internal buffers
            scene.reset()
            print("[INFO]: Resetting robot state...")
        # Apply default actions to the robot
        # -- generate actions/commands
        targets = scene["robot"].data.default_joint_pos.torch
        # -- apply action to the robot
        scene["robot"].set_joint_position_target_index(target=targets)
        # -- write data to sim
        scene.write_data_to_sim()
        # perform step
        sim.step()
        # update sim-time
        sim_time += sim_dt
        count += 1
        # update buffers
        scene.update(sim_dt)

        # print information from the sensors
        print("-------------------------------")
        print(scene["camera"])
        print("Received shape of rgb   image: ", scene["camera"].data.output["rgb"].shape)
        print("Received shape of depth image: ", scene["camera"].data.output["distance_to_image_plane"].shape)
        print("-------------------------------")
        print(scene["height_scanner"])
        print(
            "Received max height value: ",
            torch.max(scene["height_scanner"].data.ray_hits_w.torch[..., -1]).item(),
        )
        print("-------------------------------")
        print(scene["contact_forces"])
        print("Received max contact force of: ", torch.max(scene["contact_forces"].data.net_normal_forces_w).item())

def main():
    """Main function."""

    # Configure the simulation
    sim_cfg = sim_utils.SimulationCfg(dt=0.005, device=args_cli.device)
    # Launch the simulator runtime that the configuration needs
    with launch_simulation(sim_cfg, args_cli):
        # Initialize the simulation context
        sim = sim_utils.SimulationContext(sim_cfg)
        # Set main camera
        sim.set_camera_view(eye=[3.5, 3.5, 3.5], target=[0.0, 0.0, 0.0])
        # Design scene
        scene_cfg = SensorsSceneCfg(num_envs=args_cli.num_envs, env_spacing=2.0)
        scene = instantiate(scene_cfg)
        # Play the simulator
        sim.reset()
        # Now we are ready!
        print("[INFO]: Setup complete...")
        # Run the simulator
        run_simulator(sim, scene)

if __name__ == "__main__":
    # run the main function
    main()
```

## The Code Explained

Similar to the previous tutorials, where we added assets to the scene, the sensors are also added
to the scene using the scene configuration. All sensors inherit from the `sensors.SensorBase` class
and are configured through their respective config classes. Each sensor instance can define its own
update period, which is the frequency at which the sensor is updated. The update period is specified
in seconds through the `sensors.SensorBaseCfg.update_period` attribute.

Depending on the specified path and the sensor type, the sensors are attached to the prims in the scene.
They may have an associated prim that is created in the scene or they may be attached to an existing prim.
For instance, the camera sensor has a corresponding prim that is created in the scene, whereas for the
contact sensor, activating the contact reporting is a property on a rigid body prim.

In the following, we introduce the different sensors we use in this tutorial and how they are configured.
For more description about them, please check the `sensors` module.

### Camera sensor

A camera is defined using the `sensors.CameraCfg`. It is based on the USD Camera sensor and
the different data types are captured using Omniverse Replicator API. Since it has a corresponding prim
in the scene, the prims are created in the scene at the specified prim path.

The configuration of the camera sensor includes the following parameters:

* `sensors.CameraCfg.spawn`: The type of USD camera to create. This can be either
  `~sim.spawners.sensors.PinholeCameraCfg` or `~sim.spawners.sensors.FisheyeCameraCfg`.
* `sensors.CameraCfg.offset`: The offset of the camera sensor from the parent prim.
* `sensors.CameraCfg.data_types`: The data types to capture. This can be `rgb`,
  `distance_to_image_plane`, `normals`, or other types supported by the USD Camera sensor.

To attach an RGB-D camera sensor to the head of the robot, we specify an offset relative to the base
frame of the robot. The offset is specified as a translation and rotation relative to the base frame,
and the `sensors.CameraCfg.OffsetCfg.convention` in which the offset is specified.

In the following, we show the configuration of the camera sensor used in this tutorial. We set the
update period to 0.1s, which means that the camera sensor is updated at 10Hz. The prim path expression is
set to `{ENV_REGEX_NS}/Robot/base/front_cam` where the `{ENV_REGEX_NS}` is the environment namespace,
`"Robot"` is the name of the robot, `"base"` is the name of the prim to which the camera is attached,
and `"front_cam"` is the name of the prim associated with the camera sensor.

```python
camera = CameraCfg(
    prim_path="{ENV_REGEX_NS}/Robot/base/front_cam",
    update_period=0.1,
    height=480,
    width=640,
    data_types=["rgb", "distance_to_image_plane"],
    spawn=sim_utils.PinholeCameraCfg(
        focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 1.0e5)
    ),
    offset=CameraCfg.OffsetCfg(pos=(0.510, 0.0, 0.015), rot=(0.5, -0.5, 0.5, -0.5), convention="ros"),
)
```

Camera sensors need the simulator's rendering extensions, also when running without a visualizer.
`~app.launch_simulation` enables them automatically when the configuration it is given contains a
camera. Since this script passes only the simulation configuration and the camera lives in the scene
configuration, the script requests camera rendering explicitly after parsing its arguments:

```python
args_cli = parser.parse_args()
# The scene has a camera sensor, which requires the rendering extensions in headless launches.
args_cli.enable_cameras = True
```

### Height scanner

The height-scanner is implemented as a virtual ray-casting sensor. Through the
`sensors.RayCasterCfg`, we specify the ray pattern. PhysX-based backends
cast against the configured meshes, while Newton casts against its live scene
BVH and ignores the mesh list. By default, `sensors.RayCasterCfg.spawn` creates
a plain USD Xform at `sensors.RayCasterCfg.prim_path` to serve as the sensor's
attachment frame. This compatibility behavior is deprecated; new configurations should
set `sensors.RayCasterCfg.spawn` to `None` and track an existing frame.

For this tutorial, the ray-cast based height scanner is attached under the base frame of the robot.
The pattern of rays is specified using the `sensors.RayCasterCfg.pattern` attribute. For
a uniform grid pattern, we specify the pattern using `~sensors.patterns.GridPatternCfg`.
Since we only care about the height information, we do not need to consider the roll and pitch
of the robot. Hence, we set the `sensors.RayCasterCfg.ray_alignment` to "yaw".

For the height-scanner, you can visualize the points where the rays hit the mesh. This is done
by setting the `sensors.SensorBaseCfg.debug_vis` attribute to true.

The entire configuration of the height-scanner is as follows:

```python
height_scanner = RayCasterCfg(
    prim_path="{ENV_REGEX_NS}/Robot/base",
    update_period=0.02,
    offset=RayCasterCfg.OffsetCfg(pos=(0.0, 0.0, 20.0)),
    ray_alignment="yaw",
    pattern_cfg=patterns.GridPatternCfg(resolution=0.1, size=[1.6, 1.0]),
    debug_vis=True,
    mesh_prim_paths=["/World/defaultGroundPlane"],
)
```

### Contact sensor

Contact sensors wrap around the PhysX contact reporting API to obtain the contact information of the robot
with the environment. Since it relies of PhysX, the contact sensor expects the contact reporting API
to be enabled on the rigid bodies of the robot. This can be done by setting the
`sim.spawners.RigidObjectSpawnerCfg.activate_contact_sensors` to true in the asset configuration.

Through the `sensors.ContactSensorCfg`, it is possible to specify the prims for which we want to
obtain the contact information. Additional flags can be set to obtain more information about
the contact, such as the contact air time, contact forces between filtered prims, etc.

In this tutorial, we attach the contact sensor to the feet of the robot. The feet of the robot are
named `"LF_FOOT"`, `"RF_FOOT"`, `"LH_FOOT"`, and `"RH_FOOT"`. We pass a Regex expression
`".*_FOOT"` to simplify the prim path specification. This Regex expression matches all prims that
end with `"_FOOT"`.

We set the update period to 0 to update the sensor at the same frequency as the simulation. Additionally,
for contact sensors, we can specify the history length of the contact information to store. For this
tutorial, we set the history length to 6, which means that the contact information for the last 6
simulation steps is stored.

The entire configuration of the contact sensor is as follows:

```python
contact_forces = ContactSensorCfg(
    prim_path="{ENV_REGEX_NS}/Robot/.*_FOOT", update_period=0.0, history_length=6, debug_vis=True
)
```

### Running the simulation loop

Similar to when using assets, the buffers and physics handles for the sensors are initialized only
when the simulation is played, i.e., it is important to call `sim.reset()` after creating the scene.

```python
# Play the simulator
sim.reset()
```

Besides that, the simulation loop is similar to the previous tutorials. The sensors are updated as part
of the scene update and they internally handle the updating of their buffers based on their update
periods.

The data from the sensors can be accessed through their `data` attribute. As an example, we show how
to access the data for the different sensors created in this tutorial:

```python
# print information from the sensors
print("-------------------------------")
print(scene["camera"])
print("Received shape of rgb   image: ", scene["camera"].data.output["rgb"].shape)
print("Received shape of depth image: ", scene["camera"].data.output["distance_to_image_plane"].shape)
print("-------------------------------")
print(scene["height_scanner"])
print(
    "Received max height value: ",
    torch.max(scene["height_scanner"].data.ray_hits_w.torch[..., -1]).item(),
)
print("-------------------------------")
print(scene["contact_forces"])
print("Received max contact force of: ", torch.max(scene["contact_forces"].data.net_normal_forces_w).item())
```

## The Code Execution

Now that we have gone through the code, let's run the script and see the result:

#### uv (Recommended)

```bash
uv run python scripts/tutorials/04_sensors/add_sensors_on_robot.py --num_envs 2 --viz kit
```

#### isaaclab.sh / isaaclab.bat

```bash
./isaaclab.sh -p scripts/tutorials/04_sensors/add_sensors_on_robot.py --num_envs 2 --viz kit
```

This command should open a stage with a ground plane, lights, and two quadrupedal robots.
Around the robots, you should see red spheres that indicate the points where the rays hit the mesh.
Additionally, you can switch the viewport to the camera view to see the RGB image captured by the
camera sensor. Please check [here](https://youtu.be/htPbcKkNMPs?feature=shared) for more information
on how to switch the viewport to the camera view.

![result of add\_sensors\_on\_robot.py](/isaaclab/_fern-img/7bb6b296dfa8d2d1653c00e016ead02af88b550e9c6b0423d55be4bae00a8026.webp)

To stop the simulation, you can either close the window, or press `Ctrl+C` in the terminal.

While in this tutorial, we went over creating and using different sensors, there are many more sensors
available in the `sensors` module. We include minimal examples of using these sensors in the
`scripts/tutorials/04_sensors` directory. For completeness, these scripts can be run using the
following commands:

#### uv (Recommended)

```bash
# Frame Transformer
uv run python scripts/tutorials/04_sensors/run_frame_transformer.py --viz kit

# Ray Caster
uv run python scripts/tutorials/04_sensors/run_ray_caster.py --viz kit

# Ray Caster Camera
uv run python scripts/tutorials/04_sensors/run_ray_caster_camera.py --viz kit

# USD Camera
uv run python scripts/tutorials/04_sensors/run_usd_camera.py --viz kit
```

#### isaaclab.sh / isaaclab.bat

```bash
# Frame Transformer
./isaaclab.sh -p scripts/tutorials/04_sensors/run_frame_transformer.py --viz kit

# Ray Caster
./isaaclab.sh -p scripts/tutorials/04_sensors/run_ray_caster.py --viz kit

# Ray Caster Camera
./isaaclab.sh -p scripts/tutorials/04_sensors/run_ray_caster_camera.py --viz kit

# USD Camera
./isaaclab.sh -p scripts/tutorials/04_sensors/run_usd_camera.py --viz kit
```