# Problem Specification Before we explore how a robot learns, let’s define the problem we want it to solve. We’ll return to our gear-assembly example. ![Multiple simulated robot arms practice the gear-assembly task in parallel in Isaac Lab.](images/gear-insertion-training.jpg) For this scene, we can start with four components. These choices determine what the policy can know, what it can change, and how you will decide whether it worked. In Isaac Lab, they become the observation, action, and task configuration rather than remaining abstract definitions. - **Observations:** information available to the robot. - **Actions:** commands that the policy can issue. - **Policies:** functions that select actions based on observations. - **Tasks:** descriptions of the environment, goals, and the robot. The **observation space** specifies the available inputs and their shapes, such as a camera image and joint angles. The **action space** specifies the possible outputs and their bounds, such as joint angle positions. A **policy** selects an action based on the current observation. A **task** describes the robot (is it an arm? a mobile robot?), the environment (what objects are present?), and the goal (what is success? what is a failure? how is a policy rewarded?). You’ll often see a policy written as a function of the current observation: $$ a_t = \pi_\theta(o_t) $$ where $o_t$ is the **observation** at time $t$, $a_t$ is the **action** selected by the **policy** $\pi_\theta$, and $\theta$ contains the learned parameters of the policy. The action changes the environment, producing the next observation ($o_{t+1}$). ## States and Observations A scene’s **state** describes the robot and its environment at a particular moment, including the physical quantities needed to predict how they respond to an action, such as positions, velocities, masses, and textures. An observation provides a potentially incomplete and noisy view of those quantities. For example, a single image (observation) may not reveal an object’s mass or velocity, which are part of the state. We’ll use $s_t$ for the environment state at time $t$, which is distinct from, but related to, $o_t$. Keeping these symbols separate matters in robot learning. A simulator may know the exact pose and velocity of every object, the contact forces, and the material properties, while a deployed robot has access only to its sensors. You can think of an observation as being produced from the state: $$ o_t \sim p(\cdot \mid s_t). $$ The observation process can omit information, add sensor noise, or transform physical quantities into another representation. A camera frame, for example, conveys information about object poses, but it does not directly include an object’s mass or velocity. Joint encoders provide a noisy measurement of a robot’s angles or velocities, but they do not directly reveal the forces acting on the robot. ![An isometric camera view of a cube contrasts with precise object information: position, orientation, linear velocity, angular velocity, mass, and friction.](images/camera-frame-vs-object-state-proposal.svg) A human operator can often infer much of the precise state information from a camera frame (or sequence of frames), but a policy must be trained to do so. It is often easier to train a policy with access to the full state, and then distill that knowledge into a policy that uses only observations. Robot learning often uses a **Markov decision process (MDP)** to describe the relationship between states, observations, actions, and rewards. An MDP is a mathematical framework for modeling sequential decision-making. Think of it as repeatedly calling the policy function in a loop. The defining *Markov assumption* is that the current state contains all information needed to predict the next-state distribution after an action. A policy whose observation does not contain the complete state faces a **partially observable** problem. Two different states can produce similar observations but require different actions. This learning path makes the distinction between state and observation concrete: 1. You will first [train a teacher policy](https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-lab/2026-10.0/train-a-robot-arm/05-teacher-policy.md) using privileged state information available only in simulation. 2. Next you will [train a camera policy](https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-lab/2026-10.0/train-a-robot-arm/06-distilled-policy.md) using wrist-camera images plus proprioceptive and controller inputs: joint positions, joint velocities, joint targets, and the previous action. The teacher and student solve the same physical task, but they do not receive the same information about the state. ## Actions and Commands The output of a policy is not necessarily a motor command. Let $a_t$ denote the policy action and $u_t$ the command ultimately sent to the actuator. Action processing can scale, offset, clip, or otherwise transform the policy output: $$ a_t \sim \pi_\theta(\cdot \mid o_t), \qquad u_t = g(a_t, s_t). $$ For example, a policy output of $0.5$ might become a joint-position target of `target = 0.5 * scale + initial_position`. A low-level joint controller then converts that target into effort. The robot and its environment respond over some period of time before the policy receives the next state or observation. $$ s_{t+1} \sim p(\cdot \mid s_t, u_t). $$ ## Review ### Check Your Understanding Can two different physical states produce the same observation? ### Answer Yes. An observation can hide differences in physical state, such as an object’s velocity or an occluded pose. Those differences can require different actions. ### Apply What You Learned How would you specify a vial-placement task for a state-based policy and a camera-based policy? Include each policy’s available and hidden information, actions, and success criterion. ### Agent Tutor Prompt Copy this prompt and include your answer: > Review my answer to this question: How would you specify a vial-placement task for a state-based policy and a camera-based policy? Include each policy’s available and hidden information, actions, and success criterion. > My answer: [paste your answer here]. > Check my reasoning about available and hidden information, actions, and success. Ask for missing context rather than assuming it. Identify what I explained well, point out one gap or misconception, and ask a follow-up question that helps me improve my answer. Let me revise it before supplying a complete answer. ### Key Takeaways - Define success, observations, actions, and episode boundaries before training. - A policy sees observations, which may omit important physical state. - Isaac Lab expresses these choices in environment configuration and MDP terms.