Isaac Sim Action Graphs

The ROS 2 Action Graphs that drive the SIL scene in Isaac Sim. Since Isaac Sim 6.0 these graphs are built at run time by Python builders (one module per graph under closed-loop-testing/isaac-sim/sil/scripts/action_graphs/) instead of being baked into the scene USD. Each builder is idempotent, reads its shape from robots.yaml / cameras.yaml, and creates a graph at a predictable /World/<Name>Graph path so operators can find it in the Stage panel.

At load time, forklift_common.strip_baked_scene_graphs() deletes any residual baked graphs (e.g. /World/ActionGraph, ROS_Forklift_Control_Graph) so the Python builders are the single source of truth and no logic runs twice.

Graphs at a glance

The per-robot graphs (control, odometry, safety) are built once per robot in robots.yaml — the <name> in the path is the robot’s name (e.g. forklift_b_ControlGraph). Each block carries an enabled flag, so any graph can be turned off without removing its config.

Build order

The builders fire from run_actor_sdg.py after setup_simulation() and before timeline.play():

Clock Publisher Graph

Publishes simulation time to /clock so every ROS 2 node in the stack shares the sim clock instead of wall-clock time. It is global (not tied to any articulation), so it lives in its own builder.

OnPlaybackTick ─> ROS2PublishClock (/clock)
ROS2Context ────┘

Config — the top-level clock: block of robots.yaml:

clock:
  enabled: true
  clock_topic: clock   # ROS2PublishClock.topicName

RTSP Multi-Camera Graph

Builds one shared OnPlaybackTick plus a pair of nodes (IsaacCreateRenderProduct + RTSPCameraHelper) for each camera in cameras.yaml. After Play, each camera self-hosts an in-process RTSP server on its own port and streams NVENC h264 (default 1920×1080) with per-frame SEI metadata.

OnPlaybackTick ─> IsaacSimulationGate ─┬─> RP_Camera    ─> RTSP_Camera    (port 8554 /camera)
                                       ├─> RP_Camera_01 ─> RTSP_Camera_01 (port 8555 /camera_01)
                                       └─> RP_Camera_02 ─> RTSP_Camera_02 (port 8556 /camera_02)

The IsaacSimulationGate (rate divider) throttles the render-loop tick down to the sim/BT rate; without it the encoder duplicates frames on the wire and the downstream DeepStream/VST pipeline backlogs.

Config — the cameras: list in cameras.yaml (name, camera_prim, port, mount_path; optional spawn: block to create the camera prim at run time). The stream URL is rtsp://${HOST_IP}:<port><mount_path>.

Forklift Control Graph

Turns velocity commands into forklift joint commands. It subscribes to the robot’s control.cmd_vel_topic (forklift_b/cmd_vel; geometry_msgs/Twist, published by the forklift-controller service, which reads the same key) and maps it through a single swivel-drive IK ScriptNode to the IsaacArticulationController.

OnPlaybackTick ─> ROS2SubscribeTwist (forklift_b/cmd_vel) ─> SwivelIK (ScriptNode) ─> ArticulationController
ROS2Context ────────────────────────────────────┘                          (targetPrim = robot)

SwivelIK ScriptNode

The IK math from the old baked ROS_Forklift_Control_Graph plus the downstream divide-by-wheel-radius and construct-array plumbing are folded into one ScriptNode (same math, far fewer fragile node-to-node connections). Build-time constants are prepended by the builder from the robot’s model block in robots.yaml (models.<model>).

# Prepended by the builder from the robots.yaml model block:
#   DRIVE_JOINT, SWIVEL_JOINT, WHEELBASE, WHEEL_RADIUS, MAX_STEER,
#   FLIP_LINEAR_X, FLIP_ANGULAR_Z
import math
MIN_COS = 1e-3
MIN_SPEED = 1e-4
def compute(db):
    lin = db.inputs.linearVelocity
    ang = db.inputs.angularVelocity
    # Vehicle-frame vs ROS-frame sign flips (control.reverse_logic.*).
    linear_x = -lin[0] if FLIP_LINEAR_X else lin[0]
    angular_z = -ang[2] if FLIP_ANGULAR_Z else ang[2]
    heading_speed = max(abs(linear_x), MIN_SPEED)
    steer = math.atan2(angular_z * WHEELBASE, heading_speed)
    steer = max(min(steer, MAX_STEER), -MAX_STEER)
    if linear_x < 0.0:                     # reverse
        steer = 0.0 if abs(angular_z) < MIN_SPEED else -steer
        drive = -heading_speed / max(math.cos(steer), MIN_COS)
    else:
        drive = heading_speed / max(math.cos(steer), MIN_COS)
    wheel_ang = drive / WHEEL_RADIUS       # m/s -> rad/s
    db.outputs.jointNames = [DRIVE_JOINT, SWIVEL_JOINT]
    db.outputs.positionCommand = [0.0, steer]      # swivel = steering angle
    db.outputs.velocityCommand = [wheel_ang, 0.0]  # drive  = wheel angular vel
    return True

The controller applies two joints: a velocity-controlled drive wheel and a position-controlled swivel (steer) wheel.

Forklift parameters

Defined once per model in robots.yaml models: (defaults shown); a robot block can override any of them:

Forklift Odometry Graph

Publishes the forklift’s odometry — where the robot thinks it is and how fast it is moving. IsaacComputeOdometry reads the chassis prim’s motion in the simulation and outputs position, orientation, and linear/angular velocity; two publishers then put that on the ROS 2 graph:

• ROS2PublishOdometry → /forklift_b/odom (nav_msgs/Odometry)
• ROS2PublishRawTransformTree → the forklift_b/odom → forklift_b/base_link TF, so two trucks do not publish the same edge onto one /tf

OnPlaybackTick ─> IsaacComputeOdometry ─┬─> ROS2PublishOdometry     (/forklift_b/odom)
ROS2Context ────────────────────────────┼─> ROS2PublishRawTransformTree (TF)
                                         └─ (chassisPrim = robot articulation)

Why it matters: downstream ROS 2 consumers (rviz, the forklift-controller’s closed-loop feedback, TF lookups) need a continuous pose + velocity estimate to know where the forklift is over time. Because this odometry is computed directly from the articulation transform in sim, it is clean pose feedback for the control loop — distinct from the Regression Testing Reporter ground truth on /gt/* (below), which exists to score perception against a known reference.

Config — the robots.yaml odometry: block:

odometry:
  enabled: true
  odom_topic: forklift_b/odom   # ROS2PublishOdometry.topicName; the controller reads it too
  tf_topic: tf                  # ROS2PublishRawTransformTree.topicName
  # odom_frame_id / base_frame_id: default <name>/odom -> <name>/base_link
# in the model block:
robot_front: [-1.0, 0.0, 0.0]   # which body axis points "forward"

robot_front tells IsaacComputeOdometry which local axis is the robot’s forward direction (the forklift asset’s forward is -X), so reported heading and velocity match the actual travel direction.

Safety Indicator Graph

Recolors the forklift’s indicator disc each tick from one of three sources. safety_indicator.source picks it, and defaults from PSF_APP:

OnPlaybackTick ─> ROS2Subscriber (/safety/is_muted) ─> Indicator (ScriptNode)
ROS2Context ────────────────────────────────────────┘

The Script Node polls the subscriber’s outputs:data each tick rather than through a connection: ROS2Subscriber creates that attribute dynamically, and a USD-authored connection to it is not reliably bound by OmniGraph. Each color_* key in the block overrides a default.

The mute topic defaults to the per-robot /<name>/safety/is_muted, which the comm-layer publishes for every robot in COMM_ROBOT_IDS; the shipped 20x20 fleet names the global /safety/is_muted explicitly. With two ATL zones (atl-dual-override.yaml), the comm-layer publishes each robot’s topic from the zone in its safety_indicator.zone (default 1).

Config — the robots.yaml safety_indicator: block (source, muted_topic, zone, color_*, indicator_prim; the disk defaults to <articulation_prim>/<indicator.parent_prim_rel>/safety_indicator from the model, and indicator_loader.py authors it, since no scene carries a baked one).

Regression Testing Reporter Ground-Truth Graph (opt-in)

Built only when run_actor_sdg.py is started with --srr-gt (default OFF). It publishes true poses on /gt/*/tf for the Regression Testing Reporter harness to score perception against:

• the forklift via ROS2PublishTransformTree → /gt/forklift/tf
• each pedestrian via ROS2PublishRawTransformTree → /gt/character_<i>/tf, fed each frame from the character’s live Fabric world transform (the IRA-spawned characters walk in Fabric only, so the USD-authored transform stays frozen at the spawn pose).

On a normal Halos run the flag is absent and this module is never imported, so there is zero effect on the default pipeline. For how the ground truth is consumed, see Architecture.

Limitations

No lift control

The graphs control drive and steering only; the forks stay at a fixed height.

Customize for a new scene

Because shape lives in YAML, most changes are config-only:

• New robot — add a block to robots.yaml (name, model, articulation_prim, spawn, and the control / odometry / safety_indicator sub-blocks you need) and a forklift-controller service naming its ROBOT_ID. No code change.
• New camera — add an entry to cameras.yaml with a unique port.
• Different forklift model — add a models: entry with its asset_path, wheelbase / wheel_radius / drive_joint / swivel_joint, and point the robot’s model at it.
• Relocated indicator mesh — set safety_indicator.indicator_prim.
• Turn a graph off — set enabled: false on that block.

See also

• Isaac Sim Configuration — scene, cameras, and motion authoring
• Forklift Controller — the service that publishes cmd_vel and consumes odometry
• Extending the Fleet — add a truck, a drive type, or a scene
• Forklift Waypoint Generator — author curved forklift routes
• Architecture — how /gt/* ground truth is scored