This guide walks through the common curriculum helper functions and terms that can be used to create flexible curricula
for RL environments in Isaac Lab. These utilities can be passed to a ~isaaclab.managers.CurriculumTermCfg
object to enable dynamic modification of reward weights and environment parameters during training.
We cover three utilities in this guide:
- The simple function modifies reward
modify_reward_weight - The term modify any environment parameters
modify_env_param - The term modify term_cfg
modify_term_cfg
# Copyright (c) 2022-2026, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).# All rights reserved.## SPDX-License-Identifier: BSD-3-Clause"""Common functions that can be used to create curriculum for the learning environment.The functions can be passed to the `isaaclab.managers.CurriculumTermCfg` object to enablethe curriculum introduced by the function."""from __future__ import annotationsimport refrom collections.abc import Sequencefrom typing import TYPE_CHECKING, Any, ClassVarimport torchfrom ...managers import CurriculumTermCfg, ManagerTermBaseif TYPE_CHECKING:from .. import ManagerBasedRLEnvclass modify_reward_weight(ManagerTermBase):"""Curriculum that modifies the reward weight based on a step-wise schedule."""def __init__(self, cfg: CurriculumTermCfg, env: ManagerBasedRLEnv):super().__init__(cfg, env)# obtain term configurationterm_name = cfg.params["term_name"]self._term_cfg = env.reward_manager.get_term_cfg(term_name)def __call__(self,env: ManagerBasedRLEnv,env_ids: Sequence[int],term_name: str,weight: float,num_steps: int,) -> float:if env.common_step_counter > num_steps:self._term_cfg.weight = weightenv.reward_manager.set_term_cfg(term_name, self._term_cfg)return self._term_cfg.weightclass modify_env_param(ManagerTermBase):"""Curriculum term for modifying an environment parameter at runtime.This term helps modify an environment parameter (or attribute) at runtime.This parameter can be any attribute of the environment, such as the physics material properties,observation ranges, or any other configurable parameter that can be accessed via a dotted path.The term uses the `address` parameter to specify the target attribute as a dotted path string.For instance, "event_manager.cfg.object_physics_material.func.material_buckets" wouldrefer to the attribute `material_buckets` in the event manager's event term "object_physics_material",which is a tensor of sampled physics material properties.The term uses the `modify_fn` parameter to specify the function that modifies the value of the target attribute.The function should have the signature:~~~pythondef modify_fn(env, env_ids, old_value, **modify_params) -> new_value | modify_env_param.NO_CHANGE:# modify the value based on the old value and the modify parametersnew_value = old_value + modify_params["value"]return new_value~~~where `env` is the learning environment, `env_ids` are the sub-environment indices,`old_value` is the current value of the target attribute, and `modify_params`are additional parameters that can be passed to the function. The function should returnthe new value to be set for the target attribute, or the special token `modify_env_param.NO_CHANGE`to indicate that the value should not be changed.At the first call to the term after initialization, it compiles getter and setter functionsfor the target attribute specified by the `address` parameter. The getter retrieves thecurrent value, and the setter writes a new value back to the attribute.This term processes getter/setter accessors for a target attribute in an(specified byas an "address" in the term configuration `cfg.params["address"]`) the first time it is called,then on each invocation reads the current value, applies a user-provided `modify_fn`,and writes back the result. Since `None` in this case can sometime be desirable valueto write, we use token, `NO_CHANGE`, as non-modification signal to this class, see usage below.Usage:~~~pythondef resample_bucket_range(env, env_id, data, static_friction_range, dynamic_friction_range, restitution_range, num_steps):if env.common_step_counter > num_steps:range_list = [static_friction_range, dynamic_friction_range, restitution_range]ranges = torch.tensor(range_list, device="cpu")new_buckets = math_utils.sample_uniform(ranges[:, 0], ranges[:, 1], (len(data), 3), device="cpu")return new_buckets# if the step counter is not reached, return NO_CHANGE to indicate no modification.# we do this instead of returning None, since None is a valid value to set.# additionally, returning the input data would not change the value but still lead# to the setter being called, which may add overhead.return mdp.modify_env_param.NO_CHANGEobject_physics_material_curriculum = CurrTerm(func=mdp.modify_env_param,params={"address": "event_manager.cfg.object_physics_material.func.material_buckets","modify_fn": resample_bucket_range,"modify_params": {"static_friction_range": [0.5, 1.0],"dynamic_friction_range": [0.3, 1.0],"restitution_range": [0.0, 0.5],"num_step": 120000,},},)~~~"""NO_CHANGE: ClassVar = object()"""Special token to indicate no change in the value to be set.This token is used to signal that the `modify_fn` did not produce a new value. It canbe returned by the `modify_fn` to indicate that the current value should remain unchanged."""def __init__(self, cfg: CurriculumTermCfg, env: ManagerBasedRLEnv):super().__init__(cfg, env)if "address" not in cfg.params:raise ValueError("The 'address' parameter must be specified in the curriculum term configuration.")self._address: str = cfg.params["address"]# store accessor functionsself._get_fn: callable = Noneself._set_fn: callable = Nonedef __del__(self):"""Destructor to clean up the compiled functions."""# clear the getter and setter functionsself._get_fn = Noneself._set_fn = Noneself._container = Noneself._last_path = None"""Operations."""def __call__(self,env: ManagerBasedRLEnv,env_ids: Sequence[int],address: str,modify_fn: callable,modify_params: dict | None = None,):# fetch the getter and setter functions if not already compiledif not self._get_fn:self._get_fn, self._set_fn = self._process_accessors(self._env, self._address)# resolve none typemodify_params = {} if modify_params is None else modify_params# get the current value of the target attributedata = self._get_fn()# modify the value using the provided functionnew_val = modify_fn(self._env, env_ids, data, **modify_params)# set the modified value back to the target attribute# note: if the modify_fn return NO_CHANGE signal, we do not invoke self.set_fnif new_val is not self.NO_CHANGE:self._set_fn(new_val)"""Helper functions."""def _process_accessors(self, root: ManagerBasedRLEnv, path: str) -> tuple[callable, callable]:"""Process and return the (getter, setter) functions for a dotted attribute path.This function resolves a dotted path string to an attribute in the given root object.The dotted path can include nested attributes, dictionary keys, and sequence indexing.For instance, the path "foo.bar[2].baz" would resolve to `root.foo.bar[2].baz`. Thisallows accessing attributes in a nested structure, such as a dictionary or a list.Args:root: The main object from which to resolve the attribute.path: Dotted path string to the attribute variable. For e.g., "foo.bar[2].baz".Returns:A tuple of two functions (getter, setter), where:the getter retrieves the current value of the attribute, andthe setter writes a new value back to the attribute."""# Turn "a.b[2].c" into ["a", ("b", 2), "c"] and store in partspath_parts: list[str | tuple[str, int]] = []for part in path.split("."):m = re.compile(r"^(\w+)\[(\d+)\]$").match(part)if m:path_parts.append((m.group(1), int(m.group(2))))else:path_parts.append(part)# Traverse the parts to find the containercontainer = rootfor container_path in path_parts[:-1]:if isinstance(container_path, tuple):# we are accessing a list elementname, idx = container_path# find underlying attributeif isinstance(container, dict):seq = container[name] # type: ignore[assignment]else:seq = getattr(container, name)# save the container for the next iterationcontainer = seq[idx]else:# we are accessing a dictionary key or an attributeif isinstance(container, dict):container = container[container_path]else:container = getattr(container, container_path)self._container = containerself._last_path = path_parts[-1] # for "a.b[2].c", this is "c", while for "a.b[2]" it is 2if isinstance(self._container, tuple):get_value = lambda: self._container[self._last_path] # noqa: E731def set_value(val):tuple_list = list(self._container)tuple_list[self._last_path] = valself._container = tuple(tuple_list)elif isinstance(self._container, (list, dict)):get_value = lambda: self._container[self._last_path] # noqa: E731def set_value(val):self._container[self._last_path] = valelif isinstance(self._container, object):get_value = lambda: getattr(self._container, self._last_path) # noqa: E731set_value = lambda val: setattr(self._container, self._last_path, val) # noqa: E731else:raise TypeError(f"Unable to build accessors for address '{path}'. Unknown type found for access variable:"f" '{type(self._container)}'. Expected a list, dict, or object with attributes.")return get_value, set_valueclass modify_term_cfg(modify_env_param):"""Curriculum for modifying a manager term configuration at runtime.This class inherits from `modify_env_param` and is specifically designed to modify the configuration of a manager term in the environment. It mainly adds the convenience of using a simplified address style that uses "s." as a prefix to refer to the manager's configuration. For instance, instead of writing "event_manager.cfg.object_physics_material.func.material_buckets", you can write "events.object_physics_material.func.material_buckets" to refer to the same term configuration. The same applies to other managers, such as "observations", "commands", "rewards", and "terminations". Internally, it replaces the first occurrence of "s." in the address with "_manager.cfg.", thus transforming the simplified address into a full manager path. Usage: .. code-block:: python def override_value(env, env_ids, data, value, num_steps): if env.common_step_counter > num_steps: return value return mdp.modify_term_cfg.NO_CHANGE command_object_pose_xrange_adr = CurrTerm( func=mdp.modify_term_cfg, params={ "address": "commands.object_pose.ranges.pos_x", # note: `_manager.cfg` is omitted"modify_fn": override_value,"modify_params": {"value": (-0.75, -0.25), "num_steps": 12000},},)"""def __init__(self, cfg, env):# initialize the parentsuper().__init__(cfg, env)# overwrite the simplified address with the full manager pathself._address = self._address.replace("s.", "_manager.cfg.", 1)class DifficultyScheduler(ManagerTermBase):"""Adaptive difficulty scheduler for curriculum learning.Each environment keeps an integer difficulty level. At episode end the level is promoted when thereward term named by `success_term_name` reports success for that environment through a stickyboolean `succeeded` buffer, and demoted otherwise unless `promotion_only` is set. The normalizedmean difficulty across environments is exposed as `difficulty_frac` for other curriculum terms,such as `initial_final_interpolate_fn`, to interpolate their targets."""def __init__(self, cfg: CurriculumTermCfg, env: ManagerBasedRLEnv):super().__init__(cfg, env)init_difficulty: int = cfg.params["init_difficulty"]self.current_difficulties = torch.full((env.num_envs,), float(init_difficulty), device=env.device)self.difficulty_frac: float = 0.0"""Mean difficulty across environments, normalized by `max_difficulty`."""def get_state(self) -> torch.Tensor:return self.current_difficultiesdef set_state(self, state: torch.Tensor) -> None:self.current_difficulties = state.clone().to(self._env.device)def __call__(self,env: ManagerBasedRLEnv,env_ids: Sequence[int],init_difficulty: int = 0,min_difficulty: int = 0,max_difficulty: int = 50,promotion_only: bool = False,success_term_name: str = "success",) -> float:# the success term must be a class-based reward exposing a per-environment boolean `succeeded` buffersucceeded = env.reward_manager.get_term_cfg(success_term_name).func.succeeded[env_ids]current = self.current_difficulties[env_ids]demoted = current if promotion_only else current - 1self.current_difficulties[env_ids] = torch.where(succeeded, current + 1, demoted).clamp(min=min_difficulty, max=max_difficulty)# Python float: the dependent curriculum terms compare and interpolate host-sideself.difficulty_frac = (torch.mean(self.current_difficulties) / max(max_difficulty, 1)).item()return self.difficulty_fracdef initial_final_interpolate_fn(env: ManagerBasedRLEnv,env_ids: Sequence[int],data: Any,initial_value: Any,final_value: Any,difficulty_term_str: str,) -> Any:"""Interpolate a term parameter between initial and final values by the current difficulty fraction.Intended as the `modify_fn` of `modify_term_cfg`. Works on arbitrarily nested lists andtuples; scalars (int and float) are interpolated at the leaves and integers stay integers.Args:env: The environment.env_ids: Environments being updated. Unused, the interpolation is shared by all environments.data: Current value of the parameter, which fixes the structure and leaf types of the result.initial_value: Value at zero difficulty.final_value: Value at maximum difficulty.difficulty_term_str: Name of the `DifficultyScheduler` curriculum term to read.Returns:The interpolated value, or `modify_env_param.NO_CHANGE` while the difficulty is below 10%."""difficulty_term: DifficultyScheduler = getattr(env.curriculum_manager.cfg, difficulty_term_str).funcfrac = difficulty_term.difficulty_frac# leave the parameter at its configured value until the curriculum has made some progressif frac < 0.1:return modify_env_param.NO_CHANGEreturn _interpolate_nested(initial_value, final_value, data, frac)def _interpolate_nested(initial: Any, final: Any, data: Any, frac: float) -> Any:"""Interpolate leaf scalars of nested sequences, preserving the container and leaf types of `data`."""if isinstance(data, Sequence) and not isinstance(data, (str, bytes)):return type(data)(_interpolate_nested(i, f, d, frac) for i, f, d in zip(initial, final, data))value = frac * (final - initial) + initialreturn int(value) if isinstance(data, int) else value
Modifying Reward Weights
The function modify_reward_weight updates the weight of a reward term after a specified number of simulation
steps. This can be passed directly as the func in a CurriculumTermCfg.
class modify_reward_weight(ManagerTermBase):"""Curriculum that modifies the reward weight based on a step-wise schedule."""def __init__(self, cfg: CurriculumTermCfg, env: ManagerBasedRLEnv):super().__init__(cfg, env)# obtain term configurationterm_name = cfg.params["term_name"]self._term_cfg = env.reward_manager.get_term_cfg(term_name)def __call__(self,env: ManagerBasedRLEnv,env_ids: Sequence[int],term_name: str,weight: float,num_steps: int,) -> float:if env.common_step_counter > num_steps:self._term_cfg.weight = weightenv.reward_manager.set_term_cfg(term_name, self._term_cfg)return self._term_cfg.weight
Usage example:
from isaaclab.managers import CurriculumTermCfgimport isaaclab.managers.mdp as mdp# After 100k steps, set the "sparse_reward" term weight to 0.5sparse_reward_schedule = CurriculumTermCfg(func=mdp.modify_reward_weight,params={"term_name": "sparse_reward","weight": 0.5,"num_steps": 100_000,})
Dynamically Modifying Environment Parameters
The class modify_env_param is a ~isaaclab.managers.ManagerTermBase subclass that lets you target any
dotted attribute path in the environment and apply a user-supplied function to compute a new value at runtime. It
handles nested attributes, dictionary keys, list or tuple indexing, and respects a NO_CHANGE sentinel if no update
is desired.
class modify_env_param(ManagerTermBase):"""Curriculum term for modifying an environment parameter at runtime.This term helps modify an environment parameter (or attribute) at runtime.This parameter can be any attribute of the environment, such as the physics material properties,observation ranges, or any other configurable parameter that can be accessed via a dotted path.The term uses the ``address`` parameter to specify the target attribute as a dotted path string.For instance, "event_manager.cfg.object_physics_material.func.material_buckets" wouldrefer to the attribute ``material_buckets`` in the event manager's event term "object_physics_material",which is a tensor of sampled physics material properties.The term uses the ``modify_fn`` parameter to specify the function that modifies the value of the target attribute.The function should have the signature:~~~pythondef modify_fn(env, env_ids, old_value, **modify_params) -> new_value | modify_env_param.NO_CHANGE:# modify the value based on the old value and the modify parametersnew_value = old_value + modify_params["value"]return new_value~~~where ``env`` is the learning environment, ``env_ids`` are the sub-environment indices,``old_value`` is the current value of the target attribute, and ``modify_params``are additional parameters that can be passed to the function. The function should returnthe new value to be set for the target attribute, or the special token ``modify_env_param.NO_CHANGE``to indicate that the value should not be changed.At the first call to the term after initialization, it compiles getter and setter functionsfor the target attribute specified by the ``address`` parameter. The getter retrieves thecurrent value, and the setter writes a new value back to the attribute.This term processes getter/setter accessors for a target attribute in an(specified byas an "address" in the term configuration ``cfg.params["address"]``) the first time it is called,then on each invocation reads the current value, applies a user-provided ``modify_fn``,and writes back the result. Since :obj:`None` in this case can sometime be desirable valueto write, we use token, ``NO_CHANGE``, as non-modification signal to this class, see usage below.Usage:~~~pythondef resample_bucket_range(env, env_id, data, static_friction_range, dynamic_friction_range, restitution_range, num_steps):if env.common_step_counter > num_steps:range_list = [static_friction_range, dynamic_friction_range, restitution_range]ranges = torch.tensor(range_list, device="cpu")new_buckets = math_utils.sample_uniform(ranges[:, 0], ranges[:, 1], (len(data), 3), device="cpu")return new_buckets# if the step counter is not reached, return NO_CHANGE to indicate no modification.# we do this instead of returning None, since None is a valid value to set.# additionally, returning the input data would not change the value but still lead# to the setter being called, which may add overhead.return mdp.modify_env_param.NO_CHANGEobject_physics_material_curriculum = CurrTerm(func=mdp.modify_env_param,params={"address": "event_manager.cfg.object_physics_material.func.material_buckets","modify_fn": resample_bucket_range,"modify_params": {"static_friction_range": [0.5, 1.0],"dynamic_friction_range": [0.3, 1.0],"restitution_range": [0.0, 0.5],"num_step": 120000,},},)~~~"""NO_CHANGE: ClassVar = object()"""Special token to indicate no change in the value to be set.This token is used to signal that the `modify_fn` did not produce a new value. It canbe returned by the `modify_fn` to indicate that the current value should remain unchanged."""def __init__(self, cfg: CurriculumTermCfg, env: ManagerBasedRLEnv):super().__init__(cfg, env)if "address" not in cfg.params:raise ValueError("The 'address' parameter must be specified in the curriculum term configuration.")self._address: str = cfg.params["address"]# store accessor functionsself._get_fn: callable = Noneself._set_fn: callable = Nonedef __del__(self):"""Destructor to clean up the compiled functions."""# clear the getter and setter functionsself._get_fn = Noneself._set_fn = Noneself._container = Noneself._last_path = None"""Operations."""def __call__(self,env: ManagerBasedRLEnv,env_ids: Sequence[int],address: str,modify_fn: callable,modify_params: dict | None = None,):# fetch the getter and setter functions if not already compiledif not self._get_fn:self._get_fn, self._set_fn = self._process_accessors(self._env, self._address)# resolve none typemodify_params = {} if modify_params is None else modify_params# get the current value of the target attributedata = self._get_fn()# modify the value using the provided functionnew_val = modify_fn(self._env, env_ids, data, **modify_params)# set the modified value back to the target attribute# note: if the modify_fn return NO_CHANGE signal, we do not invoke self.set_fnif new_val is not self.NO_CHANGE:self._set_fn(new_val)"""Helper functions."""def _process_accessors(self, root: ManagerBasedRLEnv, path: str) -> tuple[callable, callable]:"""Process and return the (getter, setter) functions for a dotted attribute path.This function resolves a dotted path string to an attribute in the given root object.The dotted path can include nested attributes, dictionary keys, and sequence indexing.For instance, the path "foo.bar[2].baz" would resolve to `root.foo.bar[2].baz`. Thisallows accessing attributes in a nested structure, such as a dictionary or a list.Args:root: The main object from which to resolve the attribute.path: Dotted path string to the attribute variable. For e.g., "foo.bar[2].baz".Returns:A tuple of two functions (getter, setter), where:the getter retrieves the current value of the attribute, andthe setter writes a new value back to the attribute."""# Turn "a.b[2].c" into ["a", ("b", 2), "c"] and store in partspath_parts: list[str | tuple[str, int]] = []for part in path.split("."):m = re.compile(r"^(\w+)\[(\d+)\]$").match(part)if m:path_parts.append((m.group(1), int(m.group(2))))else:path_parts.append(part)# Traverse the parts to find the containercontainer = rootfor container_path in path_parts[:-1]:if isinstance(container_path, tuple):# we are accessing a list elementname, idx = container_path# find underlying attributeif isinstance(container, dict):seq = container[name] # type: ignore[assignment]else:seq = getattr(container, name)# save the container for the next iterationcontainer = seq[idx]else:# we are accessing a dictionary key or an attributeif isinstance(container, dict):container = container[container_path]else:container = getattr(container, container_path)self._container = containerself._last_path = path_parts[-1] # for "a.b[2].c", this is "c", while for "a.b[2]" it is 2if isinstance(self._container, tuple):get_value = lambda: self._container[self._last_path] # noqa: E731def set_value(val):tuple_list = list(self._container)tuple_list[self._last_path] = valself._container = tuple(tuple_list)elif isinstance(self._container, (list, dict)):get_value = lambda: self._container[self._last_path] # noqa: E731def set_value(val):self._container[self._last_path] = valelif isinstance(self._container, object):get_value = lambda: getattr(self._container, self._last_path) # noqa: E731set_value = lambda val: setattr(self._container, self._last_path, val) # noqa: E731else:raise TypeError(f"Unable to build accessors for address '{path}'. Unknown type found for access variable:"f" '{type(self._container)}'. Expected a list, dict, or object with attributes.")return get_value, set_value
Usage example:
import torchfrom isaaclab.managers import CurriculumTermCfgimport isaaclab.managers.mdp as mdpdef resample_friction(env, env_ids, old_value, low, high, num_steps):# After num_steps, sample a new friction coefficient uniformlyif env.common_step_counter > num_steps:return torch.empty((len(env_ids),), device="cpu").uniform_(low, high)return mdp.modify_env_param.NO_CHANGEfriction_curriculum = CurriculumTermCfg(func=mdp.modify_env_param,params={"address": "event_manager.cfg.object_physics_material.func.material_buckets","modify_fn": resample_friction,"modify_params": {"low": 0.3,"high": 1.0,"num_steps": 120_000,}})
Modify Term Configuration
The subclass modify_term_cfg provides a more concise style address syntax, using consistent with hydra config
syntax. It otherwise behaves identically to modify_env_param.
class modify_term_cfg(modify_env_param):"""Curriculum for modifying a manager term configuration at runtime.This class inherits from :class:`modify_env_param` and is specifically designed to modify the configuration of a manager term in the environment. It mainly adds the convenience of using a simplified address style that uses "s." as a prefix to refer to the manager's configuration. For instance, instead of writing "event_manager.cfg.object_physics_material.func.material_buckets", you can write "events.object_physics_material.func.material_buckets" to refer to the same term configuration. The same applies to other managers, such as "observations", "commands", "rewards", and "terminations". Internally, it replaces the first occurrence of "s." in the address with "_manager.cfg.", thus transforming the simplified address into a full manager path. Usage: .. code-block:: python def override_value(env, env_ids, data, value, num_steps): if env.common_step_counter > num_steps: return value return mdp.modify_term_cfg.NO_CHANGE command_object_pose_xrange_adr = CurrTerm( func=mdp.modify_term_cfg, params={ "address": "commands.object_pose.ranges.pos_x", # note: `_manager.cfg` is omitted"modify_fn": override_value,"modify_params": {"value": (-0.75, -0.25), "num_steps": 12000},},)"""def __init__(self, cfg, env):# initialize the parentsuper().__init__(cfg, env)# overwrite the simplified address with the full manager pathself._address = self._address.replace("s.", "_manager.cfg.", 1)
Usage example:
def override_command_range(env, env_ids, old_value, value, num_steps):# Override after num_stepsif env.common_step_counter > num_steps:return valuereturn mdp.modify_term_cfg.NO_CHANGErange_override = CurriculumTermCfg(func=mdp.modify_term_cfg,params={"address": "commands.object_pose.ranges.pos_x","modify_fn": override_command_range,"modify_params": {"value": (-0.75, -0.25),"num_steps": 12_000,}})