# Distill a Student Policy

We’ve reached the final stage in training the SO-101 to pick up a vial and place
it in a rack. The teacher policy learned to do this using privileged information
from the simulation. A real robot cannot read the vial’s position directly from
the simulator, so let’s train a **student policy** that uses a wrist-camera
image and the robot’s joint state, both of which are readily available on the
SO-101.

![At the same simulation state, privileged state enters a frozen teacher while wrist RGB and proprioception enter a camera student. The student's action advances the environment, the teacher supplies a target action, and an action-matching loss updates only the student.](images/student-driven-distillation.svg)

Stay on `learning-path` with the preceding lesson’s changes in place. **Replace**
only the marked regions named below; imports and supporting code are supplied.
You can compare your working copy with the completed reference at any time:

```bash
git diff main -- src/
```

## Adding Camera Observations

The following configuration belongs in `camera_env_cfg.py`. This configuration
could be placed in `env_cfg.py`, but separating the camera-specific
configuration keeps each file focused.

The camera is attached to the gripper and renders an 80 by 60 overscan.
The observation term projects it to a 64 by 48 red, green, and blue (RGB) image at the same 30 Hz rate as the robot actions. Its view changes as the wrist moves.
The recording below shows the raw 80 by 60 wrist view before projection.
The policy receives a history of two processed 64 by 48 RGB frames.

![Wrist camera view of the SO-101.](images/so101-seed42-wrist-policy.mp4)

Isaac Lab has built-in support for cameras. Here we specify the camera’s
characteristics.

**Replace the contents**

- File: `src/isaaclab_tutorial/tasks/place_vial/config/so101/camera_env_cfg.py`
- Markers: `# BEGIN lesson-06-SO101CameraSceneCfg` and `# END lesson-06-SO101CameraSceneCfg`

Paste the code below between the markers. Keep both marker comments.

```python
@configclass
class SO101CameraSceneCfg(SO101SceneCfg):
    """SO-101 scene with the rectified pinhole wrist camera and handles to the randomized visual materials."""

    wrist_camera = CameraCfg(
        prim_path="{ENV_REGEX_NS}/Robot/gripper/wowrobo_2MP_camera/pinhole",
        spawn=PinholeCameraCfg(focal_length=_FOCAL_LENGTH, horizontal_aperture=_HORIZONTAL_APERTURE),
        offset=CameraCfg.OffsetCfg(convention="opengl"),
        data_types=["rgb"],
        width=80,
        height=60,
        update_period=1.0 / 30.0,
        update_latest_camera_pose=True,
        renderer_cfg=_camera_renderer_cfg(),
        background_color=(0.3, 0.3, 0.3),
    )

    # The Newton renderer only reads material color.
    robot_visual = VisualMaterialCfg(prim_path="{ENV_REGEX_NS}/Robot/Looks/material_a_d_printed", spawn=None)
    desk_visual = VisualMaterialCfg(prim_path="{ENV_REGEX_NS}/Desk/Looks/Wood", spawn=None)
    rack_visual = VisualMaterialCfg(prim_path="{ENV_REGEX_NS}/Rack/WorkshopVisual/Looks/OmniPBR", spawn=None)
    vial_body_visual = VisualMaterialCfg(prim_path="{ENV_REGEX_NS}/Vial/Looks/Vial_plastic", spawn=None)
    vial_cap_visual = VisualMaterialCfg(prim_path="{ENV_REGEX_NS}/Vial/Looks/Vial_cap", spawn=None)
    vial_label_visual = VisualMaterialCfg(prim_path="{ENV_REGEX_NS}/Vial/Looks/Label", spawn=None)

    def __post_init__(self):
        self.robot.spawn.hide_camera_housing = False
        self.vial.spawn.usd_path = str(VIAL_LABELED_USD)
```

As we do with physical characteristics (e.g., randomly adjusting the mass of the
vial), we can also randomize a camera’s characteristics. This process is called
**domain randomization**. It helps the student policy learn to ignore irrelevant
details in the camera image, such as lighting and color balance, so that it can
focus on the task of picking up the vial and placing it in the rack.

The supplied `_camera_renderer_cfg()` helper returns the renderer configuration
with shadows enabled. The `VisualMaterialCfg` entries reference existing
materials with `spawn=None`, so the optional appearance-randomization events can
find and change their colors without creating new scene objects.

**Replace the contents**

- File: `src/isaaclab_tutorial/tasks/place_vial/config/so101/camera_env_cfg.py`
- Markers: `# BEGIN lesson-06-WristImageCfg` and `# END lesson-06-WristImageCfg`

Paste the code below between the markers. Keep both marker comments.

```python
@configclass
class WristImageCfg(ObsGroup):
    """Two-frame wrist RGB history with episode-consistent photometric (and, for sim2real, intrinsics) variation."""

    image = ObsTerm(
        func=mdp.wrist_rgb,
        params={
            "sensor_cfg": SceneEntityCfg("wrist_camera"),
            "exposure_range": (0.75, 1.25),
            "contrast_range": (0.85, 1.15),
            "white_balance_range": (0.90, 1.10),
            "brightness_range": (-0.05, 0.05),
            "gamma_range": preset(default=(1.0, 1.0), sim2real=(0.85, 1.15)),
            "blur_range": preset(default=(0.0, 0.0), sim2real=(0.0, 0.5)),
            "history_length": 2,
            "projection_size": POLICY_IMAGE_SIZE,
            "focal_length_pixels": WRIST_FOCAL_LENGTH_PIXELS,
            "focal_scale_range": preset(default=(1.0, 1.0), sim2real=(0.95, 1.05)),
            "principal_point_pixels": preset(default=0.0, sim2real=1.5),
            "radial_distortion_range": preset(default=(0.0, 0.0), sim2real=(-0.04, 0.04)),
        },
        noise=UniformNoiseCfg(n_min=-0.025, n_max=0.025),
    )

    def __post_init__(self):
        self.enable_corruption = True
```

### From Camera Pixels to Policy Input

`mdp.wrist_rgb`, implemented in `mdp/observations.py`, remembers the image
settings and recent frames for each environment between calls. That makes it
a *stateful* observation term. Its `__call__` method reads the camera sensor
and prepares the policy input in four steps:

1. Read the first three image channels, convert to floating point, and divide
   by 255. Rearrange the raw batch from `(num_envs, 60, 80, 3)` to a
   channels-first layout, `(num_envs, 3, 60, 80)`, for image processing.
   Here, `num_envs` is the number of parallel environments, 3 is the number of
   color channels, and 60 and 80 are the image height and width.
2. Use `camera_sampling_grid` and PyTorch’s `grid_sample` to produce the
   policy view, 48 pixels high and 64 pixels wide. The grid tells the helper
   where to sample each output pixel from the larger rendered image. It
   accounts for changes in zoom (focal scale), image center (principal-point
   offset), and lens-like warping (radial distortion). The extra border,
   called *overscan*, supplies pixels when these changes shift the view.
3. When corruption is enabled, apply the sampled blur, gamma, contrast,
   exposure, white balance, and brightness changes, then clamp the transformed
   image to `[0, 1]`. Here, *corruption* means deliberate image variation
   during training, such as changing brightness or color balance.
4. Pass the image to `CameraFrameStack`, which keeps the two most recent RGB
   frames, oldest first. Stacking them along the channel dimension produces
   `(num_envs, 6, 48, 64)` for the configured history length of two.

The term’s `reset(env_ids)` samples new transformation parameters and clears
history only for the environments starting new episodes. Those sampled
parameters stay fixed during an episode. The first new image fills both history
slots, so the student cannot accidentally see a frame from the previous
attempt. The observation manager separately applies the `UniformNoiseCfg`
noise when corruption is enabled; that additive noise can move values outside
the helper’s `[0, 1]` range. Turning corruption off skips those blur, color,
and brightness changes and the manager’s noise. The helper still maps pixels
to the policy view and stacks the images.

### From Joint State to Proprioception

The student also receives **proprioception** (the robot’s ability to sense its
own state): joint positions, velocities, targets, and its previous action. Here
we add noise to the joint readings to simulate real sensor noise
(`self.enable_corruption = True` is needed to add the noise).

**Replace the contents**

- File: `src/isaaclab_tutorial/tasks/place_vial/config/so101/camera_env_cfg.py`
- Markers: `# BEGIN lesson-06-ProprioceptionCfg` and `# END lesson-06-ProprioceptionCfg`

Paste the code below between the markers. Keep both marker comments.

```python
@configclass
class ProprioceptionCfg(ObsGroup):
    """Proprioceptive observations available on the real robot, in the encoders' (possibly biased) coordinates."""

    joint_pos = ObsTerm(
        func=mdp.encoder_joint_pos,
        params={"asset_cfg": SceneEntityCfg("robot")},
        noise=UniformNoiseCfg(n_min=-0.01, n_max=0.01),
    )
    joint_vel = ObsTerm(
        func=mdp.joint_vel,
        params={"asset_cfg": SceneEntityCfg("robot")},
        noise=UniformNoiseCfg(n_min=-0.02, n_max=0.02),
    )
    joint_target = ObsTerm(
        func=mdp.encoder_joint_target,
        params={"asset_cfg": SceneEntityCfg("robot")},
        noise=UniformNoiseCfg(n_min=-0.005, n_max=0.005),
    )
    previous_action = ObsTerm(func=mdp.last_action)

    def __post_init__(self):
        self.enable_corruption = True
        self.concatenate_terms = True
```

`encoder_joint_pos` reads simulated joint angles and adds the calibration bias
stored by `randomize_encoder_bias`, or zero if no bias has been sampled.
`encoder_joint_target` adds that same bias to `joint_target`, keeping measured
positions and requested positions in the same encoder coordinates. You can
think of this bias as a sensor that consistently reads a little too high or
too low. The optional transfer preset samples a bias once per episode, so it
stays fixed during that attempt. The observation manager adds separate uniform
measurement noise. The four six-value terms in
`ProprioceptionCfg` form a 24-value vector, while the wrist images remain a
separate observation group.

For distillation, a separate `teacher_state` [observation
group](https://isaac-sim.github.io/IsaacLab/release/3.0.0/source/api/lab/isaaclab.managers.html#isaaclab.managers.ObservationGroupCfg)
is added to the camera environment. It lets the teacher *see* more than the
student so that it can label the student’s training states. In distillation, we
train the student to match the teacher’s behavior.

**Replace the contents**

- File: `src/isaaclab_tutorial/tasks/place_vial/config/so101/camera_env_cfg.py`
- Markers: `# BEGIN lesson-06-CameraObservationsCfg` and `# END lesson-06-CameraObservationsCfg`

Paste the code below between the markers. Keep both marker comments.

```python
@configclass
class CameraObservationsCfg:
    """Deployable student inputs, the state teacher's inputs (distillation labels), and the privileged critic."""

    wrist_rgb: WristImageCfg = WristImageCfg()
    proprioception: ProprioceptionCfg = ProprioceptionCfg()
    teacher_state: PolicyStateGroupCfg = PolicyStateGroupCfg()
    critic: CriticStateGroupCfg = CriticStateGroupCfg()
```

**Replace the contents**

- File: `src/isaaclab_tutorial/tasks/place_vial/config/so101/camera_env_cfg.py`
- Markers: `# BEGIN lesson-06-SO101VialCameraEnvCfg` and `# END lesson-06-SO101VialCameraEnvCfg`

Paste the code below between the markers. Keep both marker comments.

```python
@configclass
class SO101VialCameraEnvCfg(SO101VialEnvCfg):
    """Vial placement from wrist RGB and proprioception, distilled from the state teacher."""

    scene: SO101CameraSceneCfg = SO101CameraSceneCfg(num_envs=1024, env_spacing=0.9, replicate_physics=True)
    observations: CameraObservationsCfg = CameraObservationsCfg()
    events: EventsCfg = preset(default=EventsCfg(), sim2real=CameraSim2RealEventsCfg())

    def play_mode(self):
        from isaaclab_tutorial.utils import evaluation

        super().play_mode()
        if not evaluation.EXACT_EVALUATION_ACTIVE:
            self.scene.num_envs = min(self.scene.num_envs, 16)
```

### Identify Deployable Student Inputs

Before training, list the inputs in `wrist_rgb` and `proprioception` that the
student can obtain on the real SO-101. Which information in `teacher_state`
depends on simulation instead? Explain why `teacher_state` can label the
student’s training states but cannot be a student input on hardware.

The student receives two consecutive RGB frames, giving it short visual history.
The nominal task applies image and proprioceptive noise. The optional `sim2real`
preset adds camera mount, intrinsics, appearance, and physical variation; use it
with a teacher trained under the same physical preset.

## Distillation Training Configuration

In `rsl_rl_distillation_cfg.py`, the student uses a convolutional neural network
(CNN) to process images. CNNs are a common choice for image-based policies. The
teacher model has the same configuration as the state actor from the previous
lesson, including its multilayer perceptron (MLP), so we can load the trained
checkpoint. The CNN and MLP policies produce the same six robot actions.

**Replace the contents**

- File: `src/isaaclab_tutorial/tasks/place_vial/config/so101/agents/rsl_rl_distillation_cfg.py`
- Markers: `# BEGIN lesson-06-distillation` and `# END lesson-06-distillation`

Paste the code below between the markers. Keep both marker comments.

```python
@configclass
class SO101CameraDistillationRunnerCfg(RslRlDistillationRunnerCfg):
    """DAgger-style distillation: the student acts, the loaded state teacher labels every visited state."""

    seed = 42
    num_steps_per_env = 32
    max_iterations = 1600
    save_interval = 100
    experiment_name = "so101_vial_camera_distillation"
    run_name = ""
    obs_groups = {"student": ["wrist_rgb", "proprioception"], "teacher": ["teacher_state"]}
    clip_actions = 1.0
    student = RslRlCNNModelCfg(
        # Compact encoder for the 64 x 48 wrist image.
        cnn_cfg=RslRlCNNModelCfg.CNNCfg(
            output_channels=[16, 32, 32], kernel_size=[5, 3, 3], stride=[2, 2, 2], activation="elu"
        ),
        hidden_dims=[256, 128],
        activation="elu",
        obs_normalization=True,
        distribution_cfg=RslRlCNNModelCfg.GaussianDistributionCfg(init_std=0.05, std_type="log"),
    )
    # Must mirror the state actor so its checkpoint loads; the teacher always acts deterministically.
    teacher = RslRlMLPModelCfg(
        hidden_dims=[256, 256, 128],
        activation="elu",
        obs_normalization=True,
        distribution_cfg=RslRlMLPModelCfg.GaussianDistributionCfg(init_std=0.2, std_type="log", std_range=(0.05, 0.3)),
    )
    algorithm = RslRlDistillationAlgorithmCfg(
        class_name=(
            "isaaclab_tutorial.tasks.place_vial.config.so101.agents.rsl_rl_distillation_cfg:BoundedTeacherDistillation"
        ),
        num_learning_epochs=4,
        learning_rate=5.0e-4,
        gradient_length=1,
        max_grad_norm=1.0,
        loss_type="mse",
    )
```

During training, the student acts in the environment. The teacher sees the state
reached by those actions and provides the action it would choose. The student
learns to match those actions using the mean squared error loss shown in the
configuration (`loss_type="mse"`).

The `obs_groups` mapping routes `wrist_rgb` and `proprioception` to the student
and `teacher_state` to the teacher. The student combines image features from
its CNN with the vector observations to produce six actions. Matching the
teacher model and observation ordering to the saved state actor lets the
runner load its checkpoint correctly. Keep the inputs in their original order:
even if two vectors have the same length, swapping joint positions and
velocities would give the teacher the wrong meaning for each input.

## Registering the Camera Task

Register the camera environment and its distillation configuration with Isaac
Lab so the command-line tools can find them. This registration goes in `so101/__init__.py`.

**Replace the contents**

- File: `src/isaaclab_tutorial/tasks/place_vial/config/so101/__init__.py`
- Markers: `# BEGIN lesson-06-camera-registration` and `# END lesson-06-camera-registration`

Paste the code below between the markers. Keep both marker comments.

```python
gym.register(
    id="IsaacTutorial-Place-Vial-SO101-Camera",
    entry_point="isaaclab.envs:ManagerBasedRLEnv",
    disable_env_checker=True,
    kwargs={
        "env_cfg_entry_point": f"{_PACKAGE}.camera_env_cfg:SO101VialCameraEnvCfg",
        "rsl_rl_cfg_entry_point": f"{_PACKAGE}.agents.rsl_rl_distillation_cfg:SO101CameraDistillationRunnerCfg",
        "rsl_rl_ppo_cfg_entry_point": f"{_PACKAGE}.agents.rsl_rl_ppo_cfg:SO101CameraPPORunnerCfg",
        "default_agent": "rsl_rl",
    },
)
```

The camera task supports both distillation and PPO training without a
teacher. After completing the snippets, run the project’s tests:

```bash
uv run pytest -q
```

## Training the Student

Now let’s train the student policy. It loads the teacher checkpoint from the
previous lesson. Run the commands from the root of your cloned project.

1. Train the student policy. Use `tmux` as in the previous lesson if you want
   training to continue when your terminal disconnects. **Replace** `.../MODEL.pt`
   with the teacher checkpoint path you selected earlier:
   ```bash
   TEACHER_CHECKPOINT=logs/rsl_rl/so101_vial_state/.../MODEL.pt

   uv run isaaclab train                          \
     --rl_library rsl_rl                          \
     --task IsaacTutorial-Place-Vial-SO101-Camera \
     --num_envs 1024                              \
     --max_iterations 1600                        \
     --run_name distillation                      \
     --checkpoint "$TEACHER_CHECKPOINT"           \
     presets=newton_mjwarp,newton_renderer
   ```
2. Follow the training progress in TensorBoard (use port forwarding if you are
   running on a remote machine):
   ```bash
   uv run tensorboard --logdir logs/rsl_rl/
   ```

## Viewing the Student

Let’s watch the student in action to see how well it learned the task.

1. Find the student checkpoints.
   ```bash
   find logs/rsl_rl -path '*.pt' -print
   ```

   You should see checkpoints for both the teacher and student policies.
2. Set a student policy checkpoint to view.
   ```bash
   STUDENT_CHECKPOINT=logs/rsl_rl/so101_vial_camera_distillation/.../MODEL.pt
   ```

   **Replace** `.../MODEL.pt` with the path to the highest-numbered checkpoint in
   your latest student training run. The sample video below uses `model_1599.pt`.
3. Play the student policy.
   ```bash
   uv run --extra video                           \
     isaaclab play                                \
     --rl_library rsl_rl                          \
     --task IsaacTutorial-Place-Vial-SO101-Camera \
     --num_envs 16                                \
     --seed 42                                    \
     --checkpoint "$STUDENT_CHECKPOINT"           \
     --video --video_length 900                   \
     --visualizer newton                          \
     presets=newton_mjwarp,newton_renderer        \
     env.sim.default_visualizer_cfg.eye='[2.5,-3.0,2.5]' \
     env.sim.default_visualizer_cfg.lookat='[0.19,0.02,0.075]'
   ```

Playback starts from the home phase, before the SO-101 starts its grasp. Compare
its behavior with the teacher: does it miss the vial, lose it during
reorientation, or struggle to release it into the rack? Those observations will
tell you where the camera policy needs more work.

This command uses Newton GL for the external recording and MoviePy to save
the video. Keep both presets: `newton_mjwarp` runs the physics, and
`newton_renderer` supplies the wrist-camera images your student learned from.
For optional RTX recording and its dependency requirements, see the
[teacher lesson](https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-lab/2026-10.0/train-a-robot-arm/05-teacher-policy.md#if-rtx-recording-fails).

Here’s a reference recording of the student checkpoint `model_1599.pt`:

![Student checkpoint model_1599.pt controls sixteen SO-101 robots as they attempt to pick up vials and place them in racks.](images/so101-student-policy-16envs.mp4)

Both your command and the example clip run sixteen independent environments,
and the camera is in the same spot as in the teacher’s video. Try putting the
two videos side by side and comparing each stage: grasping, lifting, turning,
and releasing the vial into the rack.

## Evaluating the Student

Let’s measure how often the student completes the whole task from *home*. Use
the checkpoint you selected for playback above and run this command from your
tutorial project’s root.

1. Run the evaluation.
   ```bash
   mkdir -p evaluation-results
   set -o pipefail

   uv run isaaclab play                                                             \
     --rl_library rsl_rl                                                            \
     --task IsaacTutorial-Place-Vial-SO101-Camera                                   \
     --num_envs 1024 --seed 42                                                      \
     --checkpoint "$STUDENT_CHECKPOINT"                                             \
     --device cuda:0 --visualizer none                                              \
     --external_callback isaaclab_tutorial.utils.evaluation.install_episode_counter \
     presets=newton_mjwarp,newton_renderer                                          \
     2>&1 | tee evaluation-results/student.log
   ```

   The command is explained in the previous lesson.
2. When playback finishes, find the summary in your saved terminal log:
   ```bash
   grep '^SO101_EVAL_RESULT=' evaluation-results/student.log
   ```

   The JSON contains episode and success counts; success, grasp, lift, insertion,
   timeout, vial-loss, and unsafe-contact rates; mean and maximum peak
   rack-contact force; and mean time to success over successful episodes. Rates
   are fractions, so multiply by 100 to get percentages. Save each repeat in a
   new log file so you can compare runs without overwriting earlier results.

## Analyzing the Student’s Results

This comparison uses tutorial revision `9a4286b`, seed 42, and the default
physics preset for both policies. The student trained for 1,600 iterations with
1,024 environments, taking about 28 minutes on an RTX PRO 6000 Blackwell Max-Q.
It learned from the teacher’s final `model_799.pt` checkpoint.

A quick heads-up: your own results will probably differ a bit, and that’s
normal. Training involves randomness. What matters is comparing *your* teacher
and student with the same evaluation. Keep track of each policy’s checkpoint
path, seed, and project commit so you can trace your results later.

The student’s `model_1599.pt` succeeded in **1,014 of 1,024 episodes**, an
**99.02% success rate**. The teacher succeeded 1,020 times, so the student
completed 6 fewer placements: a gap of **0.59 percentage points** in this pair
of evaluations.

| Metric                                          | Teacher (`model_799.pt`)   | Student (`model_1599.pt`)   |
|-------------------------------------------------|----------------------------|-----------------------------|
| Episodes evaluated                              | 1,024                      | 1,024                       |
| Successful placements                           | 1,020                      | 1,014                       |
| Success rate                                    | 99.61%                     | 99.02%                      |
| Grasp rate                                      | 100.00%                    | 100.00%                     |
| Lift rate                                       | 100.00%                    | 99.32%                      |
| Insertion rate                                  | 99.61%                     | 99.12%                      |
| Timeout rate                                    | 0.39% (4 episodes)         | 0.98% (10 episodes)         |
| Vial-loss rate                                  | 0.00% (0 episodes)         | 0.00% (0 episodes)          |
| Mean time to success (successful episodes only) | 7.86 s                     | 8.29 s                      |
| Mean peak rack-contact force                    | 3.38 N                     | 1.77 N                      |
| Maximum rack-contact force                      | 17.61 N                    | 13.78 N                     |
| Unsafe rack-contact rate (above 20 N)           | 0.00%                      | 0.00%                       |
<!--Download the {download}\`evaluation summaries and checkpoint provenance <../_static/data/so101-evaluation-20261015.json>\`.-->

The student’s 1,014 successes, 10 timeouts, and zero vial losses account for all
1,024 attempts in this run. Ending flags are counted independently, so an
episode can have more than one. All reported failures here were timeouts.
Watch both the approach and what happens after grasping: does the student lift
the vial, keep hold while turning, and release it into the rack?

The milestones have separate definitions. `lift_rate` records whether the vial’s
lowest point cleared the top of the rack; insertion and success have their own
conditions. Use these rates to choose what to inspect in a video, rather than
subtracting them to count failures at each stage.

Both policies placed every successful vial in hole 0. The task accepts any of
four holes, but these runs do not demonstrate equal coverage of them. The
1,024 episodes repeat 128 home starts eight times; they do not represent 1,024
unique starting poses or multiple independent training seeds.

Successful student attempts took about **0.42 seconds longer** on average. That
comparison leaves out failures. The student’s lower mean contact force does not
establish equivalent handling or hardware safety. Neither run exceeded 20 N; physical
robot validation remains a separate step.

The tutorial project also supports training the camera policy from scratch
with PPO. This module follows teacher-to-student distillation; camera PPO is an
optional alternative described in the tutorial repository.

## Review

### Check Your Understanding

Does changing only the external recording view change the policy’s camera inputs?

### Answer

No. The external recording view changes what you can inspect; the policy camera supplies the observations that drive actions. Keep policy inputs fixed when comparing recording views.

### Apply What You Learned

How would you investigate one camera-student failure using teacher comparisons and wrist-camera evidence? Describe the first visible mistake, propose a cause and an alternative, and outline one controlled comparison that could distinguish them.

### <svg version="1.1" width="1.0em" height="1.0em" class="sd-octicon sd-octicon-dependabot" viewBox="0 0 16 16" aria-hidden="true"><path d="M5.75 7.5a.75.75 0 0 1 .75.75v1.5a.75.75 0 0 1-1.5 0v-1.5a.75.75 0 0 1 .75-.75Zm5.25.75a.75.75 0 0 0-1.5 0v1.5a.75.75 0 0 0 1.5 0v-1.5Z"></path><path d="M6.25 0h2A.75.75 0 0 1 9 .75V3.5h3.25a2.25 2.25 0 0 1 2.25 2.25V8h.75a.75.75 0 0 1 0 1.5h-.75v2.75a2.25 2.25 0 0 1-2.25 2.25h-8.5a2.25 2.25 0 0 1-2.25-2.25V9.5H.75a.75.75 0 0 1 0-1.5h.75V5.75A2.25 2.25 0 0 1 3.75 3.5H7.5v-2H6.25a.75.75 0 0 1 0-1.5ZM3 5.75v6.5c0 .414.336.75.75.75h8.5a.75.75 0 0 0 .75-.75v-6.5a.75.75 0 0 0-.75-.75h-8.5a.75.75 0 0 0-.75.75Z"></path></svg> Agent Tutor Prompt

Copy this prompt and include your answer:

> Review my answer to this question: How would you investigate one camera-student failure using teacher comparisons and wrist-camera evidence? Describe the first visible mistake, propose a cause and an alternative, and outline one controlled comparison that could distinguish them.

> My answer: [paste your answer here].

> Check my reasoning about matching evaluation conditions, available student inputs, preprocessing, first visible failure, and one testable explanation. Ask for missing context rather than assuming it. Identify what I explained well, point out one gap or misconception, and ask a follow-up question that helps me improve my answer. Let me revise it before supplying a complete answer.

### What Comes Next

You’ve taken the SO-101 from a scene with no agent to a teacher that learns
from simulation state and a student that acts from a wrist camera and robot
joint measurements. The student video is a useful reminder that a
working training run is only the start: we still need to check what the robot
does from home, through grasping, lifting, turning, and release.

As a next experiment, choose one stage where the student fails more often than
the teacher. Watch several attempts that start from home, describe the first
visible mistake, and decide what evidence would help explain it. For example,
if the student misses the vial, inspect what its wrist camera can see during
the approach. This gives you a concrete question to investigate as you keep
improving the policy.

For follow-up work on the same vial-placement task, see
[Train an SO-101 Robot From Sim-to-Real With NVIDIA Isaac](https://docs.nvidia.com/learning/physical-ai/sim-to-real-so-101/latest/index.html).
That module uses Isaac Lab for data collection and evaluation of
vision-language-action models such as NVIDIA Isaac GR00T, and covers transfer
to a physical SO-101.

### Key Takeaways

- The student uses wrist-camera images and robot state rather than privileged object state.
- Distillation updates the student using a frozen teacher’s action targets.
- Evaluate complete student episodes separately and treat hardware transfer as another validation step.
