# The Skill Library In Agentic Workflows in Isaac for Healthcare you drive the pipeline with natural-language prompts, not shell commands. Each prompt loads a focused *skill* from the repository. In source checkouts these live under `skills/`; when installed, they may live in your agent’s configured skills directory. Together they form a library you compose to move from raw scene to validated policy. The current repository has 20 skills: 13 for the agentic robot-learning workflow and seven for the catheter-navigation workflow. ## Learning Objectives By the end of this lesson, you’ll be able to: - **Identify** which skill drives each agentic or catheter-navigation stage. - **Run** the full pipeline with the `i4h-workflow-e2e` skill. ## Run It With a Prompt You smoke the whole pipeline by pasting one natural-language prompt to your coding agent. It loads the `i4h-workflow-e2e` skill, which chains every stage and keeps you in the loop to review each step. ![System diagram showing how each i4h workflow skill maps to a pipeline stage and the underlying agentic subprojects.](agentic-workflows/images/i4h-workflow-skills-system-diagram.svg) ```text Run end-to-end smoke pipeline for scissor pick-and-place. ``` ## What Happens Under the Hood? Every stage of the agentic workflow has its own skill. You compose them in sequence, or hand the whole sequence to the end-to-end skill. The sections below show how the library is organized and the commands that run beneath the prompt. ### Agentic Workflow Skills The table lists each skill, its stage, and an example prompt. | Skill | Stage / Purpose | Example prompt | |---------------------------------|----------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------| | `i4h-workflow` | Overview and routing | `What does the i4h workflow include, and where should I start?` | | `i4h-workflow-setup` | Verify host requirements and run setup | `Set up the i4h workflow on this machine and tell me if any host requirements are missing.` | | `i4h-workflow-create` | Create a new env from a template | `Create a new i4h environment by forking the scissor pick-and-place task for surgical tool sorting.` | | `i4h-workflow-scene-edit` | Edit an existing scene / task / camera | `Edit the scissor pick-and-place scene to replace the scissors with a red cube and save the scene.` | | `i4h-workflow-dataset-teleop` | Record teleoperated demos to HDF5 | `Record 5 keyboard teleop demos for the scissor pick-and-place task.` | | `i4h-workflow-dataset-replay` | Replay HDF5 episodes in Isaac Sim | `Replay episode 0 from my scissor pick-and-place recording in Isaac Sim.` | | `i4h-workflow-dataset-mimic` | Expand HDF5 demos with small noise | `Expand my scissor pick-and-place recording to 10 episodes with small action and state noise.` | | `i4h-workflow-dataset-annotate` | VLM success labeling / filtering | `Run the VLM annotator on my scissor pick-and-place recording and label which episodes satisfy the task.` | | `i4h-workflow-dataset-convert` | Convert HDF5 to a LeRobot dataset | `Convert my scissor pick-and-place recording into a LeRobot dataset.` | | `i4h-lerobot-viz` | Serve the LeRobot HTML visualizer | `Serve the LeRobot visualizer for my converted scissor pick-and-place dataset and show me the local URL.` | | `i4h-workflow-finetune` | Fine-tune a GR00T / openpi PI0 policy | `Fine-tune a policy for the scissor pick-and-place task on my converted dataset for a short smoke run.` | | `i4h-workflow-validate` | Evaluate a policy / checkpoint, or run scripted state-machine smokes | `Evaluate scissor pick and place with 3 episodes.` | | `i4h-workflow-e2e` | Run the full smoke pipeline | `Run the full end-to-end smoke pipeline for the scissor pick-and-place task.` | *Optional deep dive:* ```text Open one SKILL.md and walk me through how its body drives the corresponding workflows/agentic/ subproject. ``` ### Catheter Navigation Skills These seven v0.7 skills drive the separate `workflows/catheter_navigation/` tree. They cover CT preprocessing, a vasculature digital twin, digitally reconstructed radiographs (DRRs), XPBD catheter physics, an interactive fluoroscopy viewport, and smoke tests. This optional path has its own compute and patient-data prerequisites in each skill. | Skill | Purpose | Example prompt | |----------------------------------------|-------------------------------------------|------------------------------------------------------------------------------------------------| | `i4h-catheter-navigation` | Overview and routing | `What does the catheter navigation workflow include, and where should I start?` | | `i4h-catheter-navigation-setup` | Verify host, GPU, CLI, and imports | `Set up the catheter navigation workflow and report missing host or GPU requirements.` | | `i4h-catheter-navigation-digital-twin` | Preprocess CT and segment vessels | `Build a catheter navigation digital twin from my TotalSegmentator subject at .` | | `i4h-catheter-navigation-render-drr` | Render a synthetic or CT-based DRR | `Render a synthetic DRR frame for the catheter navigation workflow.` | | `i4h-catheter-navigation-viewport` | Open the interactive fluoroscopy viewport | `Open the catheter navigation viewport using my CT cache at .` | | `i4h-catheter-navigation-smoke` | Run CPU-only smoke tests | `Run the CPU-only catheter navigation smoke tests.` | | `i4h-catheter-navigation-e2e` | Run setup, digital twin, DRR, and tests | `Run the catheter navigation v0.7 smoke pipeline using my TotalSegmentator subject at .` | ### The Agentic Router and Limitations The `i4h-workflow` skill is the **router**: it runs no stage itself, it just orients you on what’s supported and which per-stage skill to invoke next. Start there (`What does the i4h workflow include...`) whenever you’re unsure. Known limitations to consider before you plan a long run: `i4h-workflow-dataset-teleop` is verified for `keyboard`, `keyboard_23d`, and `so101_leader`, while `manus-gloves` is not verified; `i4h-workflow-finetune` and `i4h-workflow-e2e` are verified for the `scissor_pick_and_place` smoke path first. *Optional deep dive:* ```text Explain how the i4h-workflow routing skill uses each skill's description to decide which one to invoke. ``` ### One Prompt Runs Everything The `i4h-workflow-e2e` skill runs setup → record → mimic → annotate/filter → replay → convert → visualize → finetune → validate from one prompt. It is an unattended smoke test of the record-to-validate loop. *Optional deep dive:* ```text Walk me through the e2e script and how it chains the stages from setup through validate, including how it skips the optional ones. ``` ## What’s Next? You now know both skill families and how each skill maps to a stage. Next, author your own in [Adding Your Own Skills](https://docs.nvidia.com/learning/physical-ai/getting-started-with-isaac-for-healthcare/latest/agentic-workflows/09-adding-your-own-skills.md).