> This page is for version 1.4.0 (Latest) (default). > For other versions, use one of these documentation indexes: > - 1.4.0 (Latest) (default): https://docs.nvidia.com/halos/outside-in-safety/1.4.0/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/halos/outside-in-safety/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/halos/outside-in-safety/_mcp/server. # Isaac Sim Action Graphs The ROS 2 Action Graphs that drive the SIL scene in Isaac Sim. Since Isaac Sim 6.0 these graphs are **built at run time by Python builders** (one module per graph under `closed-loop-testing/isaac-sim/sil/scripts/action_graphs/`) instead of being baked into the scene USD. Each builder is idempotent, reads its shape from `robots.yaml` / `cameras.yaml`, and creates a graph at a predictable `/World/Graph` path so operators can find it in the Stage panel. > **Note** > > Adding a robot or camera is a **config change, not a code change** — add a block to `robots.yaml` / `cameras.yaml` and the matching graph is built for it on the next run. At load time, `forklift_common.strip_baked_scene_graphs()` deletes any residual baked graphs (e.g. `/World/ActionGraph`, `ROS_Forklift_Control_Graph`) so the Python builders are the single source of truth and no logic runs twice. ## Graphs at a glance | Graph | USD path | Builder module | Purpose | | ---------------------------------------- | ----------------------------- | ------------------------------ | --------------------------------------------------------------------------------------------- | | Clock | `/World/ClockPublisherGraph` | `clock.py` | Publishes `/clock` (sim time) for all ROS 2 nodes | | RTSP cameras | `/World/RTSPMultiGraph` | `rtsp_cameras.py` | One h264 RTSP stream per camera in `cameras.yaml` | | Forklift control | `/World/_ControlGraph` | `forklift_control.py` | `//cmd_vel` (Twist) → swivel-IK → articulation | | Forklift odometry | `/World/_OdometryGraph` | `forklift_odometry.py` | `IsaacComputeOdometry` → `//odom` + TF tree | | Safety indicator | `/World/_SafetyGraph` | `forklift_safety_indicator.py` | Mute decision or proximity level → indicator disc color | | Proximity line | `/World/ProximityLineGraph` | `proximity_line.py` | **Opt-in** (`proximity_line.enabled`): the pair of each proximity decision drawn on the floor | | Regression Testing Reporter ground truth | `/World/SRRGraph` | `srr_ground_truth.py` | **Opt-in** (`--srr-gt`): publishes `/gt/*/tf` | The per-robot graphs (control, odometry, safety) are built once **per robot** in `robots.yaml` — the `` in the path is the robot's `name` (e.g. `forklift_b_ControlGraph`). Each block carries an `enabled` flag, so any graph can be turned off without removing its config. ## Build order The builders fire from `run_actor_sdg.py` after `setup_simulation()` and before `timeline.play()`: ![setup\_simulation fans out to five graph builders in call order, then timeline.play](/halos/outside-in-safety/_fern-files/_img/6cb06ef212c97818ecd489d9c4f6c11c6f38ea8acfc27c358b1056cec5acd0de.webp) ## Clock Publisher Graph ![Clock publisher Action Graph in the OmniGraph editor](/halos/outside-in-safety/_fern-files/_img/fdb1b9f6f9b464310225b61e94f405448d76bee01169ff8e91a6d49b853c75b8.webp) Publishes simulation time to `/clock` so every ROS 2 node in the stack shares the sim clock instead of wall-clock time. It is global (not tied to any articulation), so it lives in its own builder. ```text OnPlaybackTick ─> ROS2PublishClock (/clock) ROS2Context ────┘ ``` Config — the top-level `clock:` block of `robots.yaml`: ```yaml clock: enabled: true clock_topic: clock # ROS2PublishClock.topicName ``` ## RTSP Multi-Camera Graph ![RTSP multi-camera Action Graph in the OmniGraph editor](/halos/outside-in-safety/_fern-files/_img/4e8f14776c3314fbfd602d571cb75805387220f5e2aef76f9963c4cac1200692.webp) Builds one shared `OnPlaybackTick` plus a pair of nodes (`IsaacCreateRenderProduct` + `RTSPCameraHelper`) for **each camera** in `cameras.yaml`. After Play, each camera self-hosts an in-process RTSP server on its own port and streams NVENC h264 (default `1920×1080`) with per-frame SEI metadata. ```text OnPlaybackTick ─> IsaacSimulationGate ─┬─> RP_Camera ─> RTSP_Camera (port 8554 /camera) ├─> RP_Camera_01 ─> RTSP_Camera_01 (port 8555 /camera_01) └─> RP_Camera_02 ─> RTSP_Camera_02 (port 8556 /camera_02) ``` The `IsaacSimulationGate` (rate divider) throttles the render-loop tick down to the sim/BT rate; without it the encoder duplicates frames on the wire and the downstream DeepStream/VST pipeline backlogs. Config — the `cameras:` list in `cameras.yaml` (`name`, `camera_prim`, `port`, `mount_path`; optional `spawn:` block to create the camera prim at run time). The stream URL is `rtsp://${HOST_IP}:`. ## Forklift Control Graph ![Forklift control Action Graph in the OmniGraph editor](/halos/outside-in-safety/_fern-files/_img/0e7f535630e9e218f3f7bd59b0e7baa862bb87edc1d099b2e7d1b69b3aefbb29.webp) Turns velocity commands into forklift joint commands. It **subscribes to the robot's `control.cmd_vel_topic`** (`forklift_b/cmd_vel`; `geometry_msgs/Twist`, published by the `forklift-controller` service, which reads the same key) and maps it through a single swivel-drive IK `ScriptNode` to the `IsaacArticulationController`. ```text OnPlaybackTick ─> ROS2SubscribeTwist (forklift_b/cmd_vel) ─> SwivelIK (ScriptNode) ─> ArticulationController ROS2Context ────────────────────────────────────┘ (targetPrim = robot) ``` > **Note** > > **Where does the motion come from?** The control graph is *reactive* — it only converts whatever `cmd_vel` it receives. The route itself is authored outside Isaac Sim: the `forklift-controller` service follows a waypoint / segment path and publishes `cmd_vel`. See [Forklift Controller](/halos/outside-in-safety/testing/sil/components/forklift-controller) for how it follows that path, and the [Forklift Waypoint Generator](/halos/outside-in-safety/testing/sil/components/isaac-sim#forklift-waypoint-generator) for authoring curved routes. ### SwivelIK ScriptNode The IK math from the old baked `ROS_Forklift_Control_Graph` plus the downstream *divide-by-wheel-radius* and *construct-array* plumbing are folded into one ScriptNode (same math, far fewer fragile node-to-node connections). Build-time constants are prepended by the builder from the robot's model block in `robots.yaml` (`models.`). ```python # Prepended by the builder from the robots.yaml model block: # DRIVE_JOINT, SWIVEL_JOINT, WHEELBASE, WHEEL_RADIUS, MAX_STEER, # FLIP_LINEAR_X, FLIP_ANGULAR_Z import math MIN_COS = 1e-3 MIN_SPEED = 1e-4 def compute(db): lin = db.inputs.linearVelocity ang = db.inputs.angularVelocity # Vehicle-frame vs ROS-frame sign flips (control.reverse_logic.*). linear_x = -lin[0] if FLIP_LINEAR_X else lin[0] angular_z = -ang[2] if FLIP_ANGULAR_Z else ang[2] heading_speed = max(abs(linear_x), MIN_SPEED) steer = math.atan2(angular_z * WHEELBASE, heading_speed) steer = max(min(steer, MAX_STEER), -MAX_STEER) if linear_x < 0.0: # reverse steer = 0.0 if abs(angular_z) < MIN_SPEED else -steer drive = -heading_speed / max(math.cos(steer), MIN_COS) else: drive = heading_speed / max(math.cos(steer), MIN_COS) wheel_ang = drive / WHEEL_RADIUS # m/s -> rad/s db.outputs.jointNames = [DRIVE_JOINT, SWIVEL_JOINT] db.outputs.positionCommand = [0.0, steer] # swivel = steering angle db.outputs.velocityCommand = [wheel_ang, 0.0] # drive = wheel angular vel return True ``` The controller applies two joints: a **velocity**-controlled drive wheel and a **position**-controlled swivel (steer) wheel. ### Forklift parameters Defined once per model in `robots.yaml` `models:` (defaults shown); a robot block can override any of them: | Parameter | Value | Usage | | ---------------------- | ------------------- | ------------------------------------- | | `wheelbase` | 1.49 m | Steering kinematics (`atan2`) | | `wheel_radius` | 0.15 m | Convert drive m/s → wheel rad/s | | `max_steer_deg` | 45° | Steering-angle clamp | | `drive_joint` | `back_wheel_drive` | Velocity command | | `swivel_joint` | `back_wheel_swivel` | Position command (steering) | | `reverse_logic.flip_*` | `true` / `true` | Negate `cmd_vel` linear.x / angular.z | ## Forklift Odometry Graph ![Forklift odometry Action Graph in the OmniGraph editor](/halos/outside-in-safety/_fern-files/_img/9828780957001774f09ad1836a9c7d244ab2377ce4c637531c42b53c0a15bfe2.webp) Publishes the forklift's **odometry** — where the robot thinks it is and how fast it is moving. `IsaacComputeOdometry` reads the chassis prim's motion in the simulation and outputs position, orientation, and linear/angular velocity; two publishers then put that on the ROS 2 graph: * `ROS2PublishOdometry` → `/forklift_b/odom` (`nav_msgs/Odometry`) * `ROS2PublishRawTransformTree` → the `forklift_b/odom` → `forklift_b/base_link` TF, so two trucks do not publish the same edge onto one `/tf` ```text OnPlaybackTick ─> IsaacComputeOdometry ─┬─> ROS2PublishOdometry (/forklift_b/odom) ROS2Context ────────────────────────────┼─> ROS2PublishRawTransformTree (TF) └─ (chassisPrim = robot articulation) ``` Why it matters: downstream ROS 2 consumers (rviz, the `forklift-controller`'s closed-loop feedback, TF lookups) need a continuous pose + velocity estimate to know where the forklift is over time. Because this odometry is computed directly from the articulation transform in sim, it is clean pose feedback for the control loop — distinct from the [Regression Testing Reporter ground truth](/halos/outside-in-safety/testing/sil/components/action-graph#regression-testing-reporter-ground-truth-graph-opt-in) on `/gt/*` (below), which exists to *score perception* against a known reference. Config — the `robots.yaml` `odometry:` block: ```yaml odometry: enabled: true odom_topic: forklift_b/odom # ROS2PublishOdometry.topicName; the controller reads it too tf_topic: tf # ROS2PublishRawTransformTree.topicName # odom_frame_id / base_frame_id: default /odom -> /base_link # in the model block: robot_front: [-1.0, 0.0, 0.0] # which body axis points "forward" ``` `robot_front` tells `IsaacComputeOdometry` which local axis is the robot's forward direction (the forklift asset's forward is `-X`), so reported heading and velocity match the actual travel direction. ## Safety Indicator Graph ![Safety indicator Action Graph in the OmniGraph editor](/halos/outside-in-safety/_fern-files/_img/bc7191654ead93b01f0ede42a0e08fe2038ca970d2fc7fab961456d081ea5511.webp) Recolors the forklift's indicator disc each tick from one of three sources. `safety_indicator.source` picks it, and defaults from `PSF_APP`: | Source | Subscribes to | Colors | | ------------------- | -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `mute` (`atl`) | `safety_indicator.muted_topic` (`std_msgs/Bool`) | **green** `(0.0, 1.0, 0.0)` muted (loading allowed), **red/orange** `(1.0, 0.3, 0.0)` alarm | | `proximity` (`pxc`) | `//proximity/state` (`std_msgs/String`), the level the forklift-controller applies, holds included | **green** NORMAL, **orange** `(1.0, 0.1, 0.0)` REDUCE, **red** `(1.0, 0.0, 0.0)` STOP, **grey** before the first decision or with proximity disabled | | `both` (`both`) | both of the above | red / orange while the truck is stopped / slowed for a person, otherwise the mute colors with the alarm in **yellow** `(1.0, 0.85, 0.0)` | ```text OnPlaybackTick ─> ROS2Subscriber (/safety/is_muted) ─> Indicator (ScriptNode) ROS2Context ────────────────────────────────────────┘ ``` The Script Node polls the subscriber's `outputs:data` each tick rather than through a connection: `ROS2Subscriber` creates that attribute dynamically, and a USD-authored connection to it is not reliably bound by OmniGraph. Each `color_*` key in the block overrides a default. The mute topic defaults to the per-robot `//safety/is_muted`, which the comm-layer publishes for every robot in `COMM_ROBOT_IDS`; the shipped 20x20 fleet names the global `/safety/is_muted` explicitly. With two ATL zones (`atl-dual-override.yaml`), the comm-layer publishes each robot's topic from the zone in its `safety_indicator.zone` (default 1). Config — the `robots.yaml` `safety_indicator:` block (`source`, `muted_topic`, `zone`, `color_*`, `indicator_prim`; the disk defaults to `//safety_indicator` from the model, and `indicator_loader.py` authors it, since no scene carries a baked one). > **Info** > > **Difference from the VST overlay**: this indicator is a **3D mesh in the simulation scene**, visible to the cameras as part of the environment. It is separate from the VST/VIOS overlay (proximity bubble, Standard/Efficient Mode text) that VIOS renders on top of the video stream in the VST UI. The VST overlay is configured in VIOS config files and provides feedback in the monitoring interface; this indicator represents a simulated physical alarm light on the forklift. ## Regression Testing Reporter Ground-Truth Graph (opt-in) ![Regression Testing Reporter ground-truth Action Graph in the OmniGraph editor](/halos/outside-in-safety/_fern-files/_img/592a6a1e2800ba930bca567a52057e778f6e87377c07c64d9a775c940fcb42c4.webp) Built **only** when `run_actor_sdg.py` is started with `--srr-gt` (default OFF). It publishes true poses on `/gt/*/tf` for the Regression Testing Reporter harness to score perception against: * the **forklift** via `ROS2PublishTransformTree` → `/gt/forklift/tf` * each **pedestrian** via `ROS2PublishRawTransformTree` → `/gt/character_/tf`, fed each frame from the character's live Fabric world transform (the IRA-spawned characters walk in Fabric only, so the USD-authored transform stays frozen at the spawn pose). On a normal Halos run the flag is absent and this module is never imported, so there is zero effect on the default pipeline. For how the ground truth is consumed, see [Architecture](/halos/outside-in-safety/testing/srr/architecture). ## Limitations ### No lift control The graphs control drive and steering only; the forks stay at a fixed height. ## Customize for a new scene Because shape lives in YAML, most changes are config-only: * **New robot** — add a block to `robots.yaml` (`name`, `model`, `articulation_prim`, `spawn`, and the `control` / `odometry` / `safety_indicator` sub-blocks you need) and a forklift-controller service naming its `ROBOT_ID`. No code change. * **New camera** — add an entry to `cameras.yaml` with a unique `port`. * **Different forklift model** — add a `models:` entry with its `asset_path`, `wheelbase` / `wheel_radius` / `drive_joint` / `swivel_joint`, and point the robot's `model` at it. * **Relocated indicator mesh** — set `safety_indicator.indicator_prim`. * **Turn a graph off** — set `enabled: false` on that block. ## See also * [Isaac Sim Configuration](/halos/outside-in-safety/testing/sil/components/isaac-sim) — scene, cameras, and motion authoring * [Forklift Controller](/halos/outside-in-safety/testing/sil/components/forklift-controller) — the service that publishes `cmd_vel` and consumes odometry * [Extending the Fleet](/halos/outside-in-safety/testing/sil/extending-the-fleet) — add a truck, a drive type, or a scene * [Forklift Waypoint Generator](/halos/outside-in-safety/testing/sil/components/isaac-sim#forklift-waypoint-generator) — author curved forklift routes * [Architecture](/halos/outside-in-safety/testing/srr/architecture) — how `/gt/*` ground truth is scored