# Managing Experiments You’ve trained a teacher and distilled a camera student. As you try to improve them, keep enough information to explain what changed and repeat the result. You don’t need a complicated experiment system to start: a clear run name, saved configuration, and a few notes go a long way. ![Repeat a baseline and a longer-training experiment with three seeds each, keeping the shared setup fixed. Compare all runs with training curves and playback from home, then save the evidence.](images/managing-experiments.svg) Compare all runs and save the evidence: TensorBoard or W&B curves and the same playback procedure from home. Keep checkpoints, configuration, and notes on successful and failed attempts. ## Optional: Repeat an Experiment Start with one question, such as whether a longer student training run reduces missed grasps. Keep the teacher checkpoint, environment settings, and evaluation procedure fixed while changing the training budget. Watch several attempts from the home phase and compare completed placements as well as the training curves. One training run may look better simply because it had a favorable random seed. To replicate an experiment, repeat it with a few different seeds while keeping the other settings fixed. For example, repeat the teacher training command: ```bash for seed in 42 43 44; do uv run isaaclab train \ --rl_library rsl_rl \ --task IsaacTutorial-Place-Vial-SO101 \ --num_envs 4096 \ --seed "$seed" \ --run_name "state-teacher-seed${seed}" \ presets=newton_mjwarp done ``` These are three full training runs, so plan for the additional compute time. Compare all three results, including failures, rather than reporting only the best one. Resuming a checkpoint continues an existing run; it does not create an independent replicate. For student experiments, keep the same teacher checkpoint across seeds so you can isolate variation in student training. Save the complete launch command, Git revision, dependency lockfile, seed, and backend choices alongside your notes. Keep the run directory under `logs/rsl_rl/`, including its saved parameters and the checkpoint you evaluated. For a student run, also record the exact teacher checkpoint. Reusing a seed helps repeat a setup, but changes in hardware or software can still affect the result. ## Optional: Track Runs With Weights & Biases TensorBoard is enough for the first lessons. **Weights & Biases (W&B)** is useful when you want a shared dashboard for comparing more runs. Isaac Lab’s RSL-RL runner supports W&B as an alternative logger through its [`logger` and `wandb_project` settings](https://isaac-sim.github.io/IsaacLab/release/3.0.0/source/api/lab_rl/isaaclab_rl.html#isaaclab_rl.rsl_rl.RslRlOnPolicyRunnerCfg). If you’d like to track your runs with W&B, sign in before starting training. The `--extra wandb` option installs the tutorial’s optional W&B dependencies. The compatibility constraint is declared in the tutorial’s `pyproject.toml`. Use this extra when signing in and training: ```bash uv run --extra wandb wandb login ``` Then add the logger settings to the teacher command: ```bash uv run --extra wandb isaaclab train \ --rl_library rsl_rl \ --task IsaacTutorial-Place-Vial-SO101 \ --num_envs 4096 \ --seed 42 \ --run_name state-teacher-seed42 \ presets=newton_mjwarp \ agent.logger=wandb \ agent.wandb_project=so101-vial ``` Add `--extra wandb` and the same logger settings to the student command. Use distinct run names and record which teacher each student used. Compare curves over the same training budget, and include playback observations in your notes: a higher reward does not necessarily mean more successful placements. Keep local checkpoints and configuration files as well as the dashboard. ## Keep Track of Backends and Presets A **backend** is the software that implements simulation physics or rendering. A **preset** selects a named configuration so you can use a supported setup without editing Python scripts. The module commands use `newton_mjwarp` for Newton physics with the MuJoCo-Warp solver, and add `newton_renderer` for the student’s camera images. These choices are part of the experiment configuration. Keep the same physics and camera-rendering choices during training and playback when checking a result. In particular, the student’s `newton_renderer` supplies its observations; `--visualizer newton` selects the external Newton GL view saved in your recording. Keep `--extra video` to encode the video. The optional RTX alternative changes only that external view, not the images the policy receives, and requires compatible ovrtx plugins. Before trying another backend or preset, check what the task supports: ```bash uv run isaaclab train \ --rl_library rsl_rl \ --task IsaacTutorial-Place-Vial-SO101-Camera \ --help ``` Treat a backend change as a separate experiment: physics and image differences can change behavior as well as speed. See Isaac Lab’s [Backends and Presets](https://isaac-sim.github.io/IsaacLab/release/3.0.0/source/concepts/backends_and_presets.html) for selector details and supported configuration choices. With those habits in place, choose one visible student failure, test one change, and keep the evidence that tells you whether it helped. Continue to [Transfer Policies From Simulation to Reality](https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-lab/2026-10.0/transfer-policies-from-simulation-to-reality/index.md) to explore the next set of challenges. ## Review ### Check Your Understanding Why should an experiment record identify the exact checkpoint? ### Answer The exact checkpoint connects the reported result to the policy that was evaluated. Recording it alongside the command and conditions helps reproduce the comparison. ### Apply What You Learned How would you plan and record a comparison that tests one change to your task or training setup? State your hypothesis, changed factor, fixed conditions, outcome measure, and the artifacts needed to reproduce the comparison. ### Agent Tutor Prompt Copy this prompt and include your answer: > Review my answer to this question: How would you plan and record a comparison that tests one change to your task or training setup? State your hypothesis, changed factor, fixed conditions, outcome measure, and the artifacts needed to reproduce the comparison. > My answer: [paste your answer here]. > Check my reasoning about hypothesis, one changed factor, fixed conditions, source and dependency versions, budget, checkpoints, outcome measure, and saved artifacts. Ask for missing context rather than assuming it. Identify what I explained well, point out one gap or misconception, and ask a follow-up question that helps me improve my answer. Let me revise it before supplying a complete answer. ### Key Takeaways - Keep source revisions, settings, checkpoints, and results together. - Change one factor when testing a hypothesis. - Compare like-for-like evaluation conditions rather than only training curves.