Introduction

Welcome to the course, Training Healthcare Robots from Scratch Using Isaac for Healthcare!

Autonomous robots are increasingly relevant in healthcare due to their ability to address workforce shortages, enhance efficiency, and improve patient outcomes. By automating routine, physically demanding, and high-risk tasks, robots support medical staff and enable more focused patient care.

This course will introduce you to NVIDIA Isaac for Healthcare and its workflows. If you’ve used Isaac Sim and Isaac Lab before you’re well prepared to study this course.

We will explain and perform a complete, end-to-end implementation that demonstrates how to build, simulate, and deploy robotic system for a specific medical application: an ultrasound scan.

Learning Objectives

By the end of this course, you will be able to:

• Create a scene in Isaac Sim with custom medical assets

• Modify a robotic arm to hold an ultrasound probe

• Generate synthetic patient CT data using MAISI CT

• Use a state machine to collect data for imitation learning

• Fine-tune GR00T N1, a Vison-Language-Action (VLA) model

• Deploy GR00T N1 to autonomously scan a liver model

If you’re new to robotics, we recommend going to our Robotics Fundamentals Learning Path, and taking the courses under Getting Started with Isaac Sim and Getting Started with Isaac Lab, then coming back to this course.

• Robotics Meets Healthcare
  • Healthcare Requires Dynamic, Adaptable Robot Behavior
    • Unique Challenges
    • Train in Isaac Lab, Simulate and Evaluate in Isaac Sim
  • Isaac for Healthcare
• Foundation Models in Healthcare Robotics
  • GR00T and Cosmos
  • Synthetic Data Generation (SDG)
  • Phantom Models