🎨 Data Designer Tutorial: Seeding Synthetic Data Generation with an External Dataset

📚 What you'll learn

In this notebook, we will demonstrate how to seed synthetic data generation in Data Designer with an external dataset.

If this is your first time using Data Designer, we recommend starting with the first notebook in this tutorial series.

📦 Import Data Designer

data_designer.config provides access to the configuration API.
DataDesigner is the main interface for data generation.

Python

1 import data_designer.config as dd
2 from data_designer.interface import DataDesigner
3

⚙️ Initialize the Data Designer interface

DataDesigner is the main object responsible for managing the data generation process.
When initialized without arguments, the default model providers are used.

Python

1 data_designer = DataDesigner()
2

🎛️ Define model configurations

Each ModelConfig defines a model that can be used during the generation process.
The "model alias" is used to reference the model in the Data Designer config (as we will see below).
The "model provider" is the external service that hosts the model (see the model config docs for more details).
By default, we use build.nvidia.com as the model provider.

Python

1 # This name is set in the model provider configuration.
2 MODEL_PROVIDER = "nvidia"
3  
4 # The model ID is from build.nvidia.com.
5 MODEL_ID = "nvidia/nemotron-3.5-lightning-30b-a3b"
6  
7 # We choose this alias to be descriptive for our use case.
8 MODEL_ALIAS = "nemotron-lightning"
9  
10 model_configs = [
11     dd.ModelConfig(
12         alias=MODEL_ALIAS,
13         model=MODEL_ID,
14         provider=MODEL_PROVIDER,
15         inference_parameters=dd.ChatCompletionInferenceParams(
16             temperature=1.0,
17             top_p=0.95,
18             max_tokens=2048,
19             extra_body={"chat_template_kwargs": {"enable_thinking": False}},
20         ),
21     )
22 ]
23

🏗️ Initialize the Data Designer Config Builder

The Data Designer config defines the dataset schema and generation process.
The config builder provides an intuitive interface for building this configuration.
The list of model configs is provided to the builder at initialization.

Python

1 config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
2

🏥 Prepare a seed dataset

For this notebook, we'll create a synthetic dataset of patient notes.
We will seed the generation process with a symptom-to-diagnosis dataset.
The notebook downloads the source CSV before generation.

🌱 Why use a seed dataset?

Seed datasets let you steer the generation process by providing context that is specific to your use case.

Seed datasets are also an excellent way to inject real-world diversity into your synthetic data.

During generation, prompt templates can reference any of the seed dataset fields.

Python

1 # Download sample dataset from Github
2 import urllib.request
3  
4 url = "https://raw.githubusercontent.com/NVIDIA/GenerativeAIExamples/refs/heads/main/nemo/NeMo-Data-Designer/data/gretelai_symptom_to_diagnosis.csv"
5 local_filename, _ = urllib.request.urlretrieve(url, "gretelai_symptom_to_diagnosis.csv")
6  
7 # Seed datasets are passed as reference objects to the config builder.
8 seed_source = dd.LocalFileSeedSource(path=local_filename)
9  
10 config_builder.with_seed_dataset(seed_source)
11

Output

DataDesignerConfigBuilder(
    seed_dataset: local seed
)

🎨 Designing our synthetic patient notes dataset

The prompt template can reference fields from our seed dataset:
- {{ diagnosis }} - the medical diagnosis from the seed data
- {{ patient_summary }} - the symptom description from the seed data

Python

1 config_builder.add_column(
2     dd.SamplerColumnConfig(
3         name="patient_sampler",
4         sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
5         params=dd.PersonFromFakerSamplerParams(),
6     )
7 )
8  
9 config_builder.add_column(
10     dd.SamplerColumnConfig(
11         name="doctor_sampler",
12         sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
13         params=dd.PersonFromFakerSamplerParams(),
14     )
15 )
16  
17 config_builder.add_column(
18     dd.SamplerColumnConfig(
19         name="patient_id",
20         sampler_type=dd.SamplerType.UUID,
21         params=dd.UUIDSamplerParams(
22             prefix="PT-",
23             short_form=True,
24             uppercase=True,
25         ),
26     )
27 )
28  
29 config_builder.add_column(dd.ExpressionColumnConfig(name="first_name", expr="{{ patient_sampler.first_name }}"))
30  
31 config_builder.add_column(dd.ExpressionColumnConfig(name="last_name", expr="{{ patient_sampler.last_name }}"))
32  
33 config_builder.add_column(dd.ExpressionColumnConfig(name="dob", expr="{{ patient_sampler.birth_date }}"))
34  
35 config_builder.add_column(
36     dd.SamplerColumnConfig(
37         name="symptom_onset_date",
38         sampler_type=dd.SamplerType.DATETIME,
39         params=dd.DatetimeSamplerParams(start="2024-01-01", end="2024-12-31"),
40     )
41 )
42  
43 config_builder.add_column(
44     dd.SamplerColumnConfig(
45         name="date_of_visit",
46         sampler_type=dd.SamplerType.TIMEDELTA,
47         params=dd.TimeDeltaSamplerParams(dt_min=1, dt_max=30, reference_column_name="symptom_onset_date"),
48     )
49 )
50  
51 config_builder.add_column(dd.ExpressionColumnConfig(name="physician", expr="Dr. {{ doctor_sampler.last_name }}"))
52  
53 config_builder.add_column(
54     dd.LLMTextColumnConfig(
55         name="physician_notes",
56         prompt="""\
57 You are a primary-care physician who just had an appointment with {{ first_name }} {{ last_name }},
58 who has been struggling with symptoms from {{ diagnosis }} since {{ symptom_onset_date }}.
59 The date of today's visit is {{ date_of_visit }}.
60  
61 {{ patient_summary }}
62  
63 Write careful notes about your visit with {{ first_name }},
64 as Dr. {{ doctor_sampler.first_name }} {{ doctor_sampler.last_name }}.
65  
66 Format the notes as a busy doctor might.
67 Respond with only the notes, no other text.
68 """,
69         model_alias=MODEL_ALIAS,
70     )
71 )
72  
73 data_designer.validate(config_builder)
74

Output

[17:53:07] [INFO] ✅ Validation passed

🔁 Iteration is key – preview the dataset!

Use the preview method to generate a sample of records quickly.
Inspect the results for quality and format issues.
Adjust column configurations, prompts, or parameters as needed.
Re-run the preview until satisfied.

Python

1 preview = data_designer.preview(config_builder, num_records=2)
2

Output

[17:53:07] [INFO] 🖼️ Preview generation in progress[17:53:07] [INFO]   |-- 🔒 Jinja rendering engine: secure[17:53:07] [INFO] ✅ Validation passed[17:53:07] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph[17:53:07] [INFO] Skipping model health checks because DATA_DESIGNER_SKIP_MODEL_HEALTH_CHECKS=1[17:53:07] [INFO] ⚡ Using async task-queue preview[17:53:07] [INFO] 📝 llm-text model config for column 'physician_notes'[17:53:07] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'[17:53:07] [INFO]   |-- model alias: 'nemotron-lightning'[17:53:07] [INFO]   |-- model provider: 'nvidia'[17:53:07] [INFO]   |-- inference parameters:[17:53:07] [INFO]   |  |-- generation_type=chat-completion[17:53:07] [INFO]   |  |-- max_parallel_requests=4[17:53:07] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}[17:53:07] [INFO]   |  |-- temperature=1.00[17:53:07] [INFO]   |  |-- top_p=0.95[17:53:07] [INFO]   |  |-- max_tokens=2048[17:53:07] [INFO] ⚡️ Async generation: 1 column(s) (column 'physician_notes'), 2 tasks across 1 row group(s)[17:53:07] [INFO] 🚀 (1/1) Dispatching with 2 records[17:53:07] [INFO] 🎲 (1/1) Preparing samplers to generate 2 records across 5 columns[17:53:07] [INFO] 🌱 (1/1) Sampling 2 records from seed dataset[17:53:07] [INFO]   |-- seed dataset size: 820 records[17:53:07] [INFO]   |-- sampling strategy: ordered[17:53:08] [INFO] 🧩 (1/1) Generating column `dob` from expression[17:53:08] [INFO] 🧩 (1/1) Generating column `first_name` from expression[17:53:08] [INFO] 🧩 (1/1) Generating column `last_name` from expression[17:53:08] [INFO] 🧩 (1/1) Generating column `physician` from expression[17:54:08] [WARNING] Observed retryable model-task error: kind=timeout; the row task will be deferred.[17:54:08] [INFO] 🔄 (1/1) Salvaging 2 deferred task(s)[17:54:13] [INFO] 📊 Progress [65.8s]:[17:54:13] [INFO]   |-- 🚗 column 'physician_notes': 1/2 (50%) 0.0 rec/s[17:54:20] [INFO] 📊 Progress [73.0s]:[17:54:20] [INFO]   |-- 🚀 column 'physician_notes': 2/2 (100%) 0.0 rec/s[17:54:20] [INFO] ✅ Async generation complete [73.0s]: 2 ok, 0 failed across 1 column(s)[17:54:20] [INFO] 📊 Model usage summary:[17:54:20] [INFO]   |-- model: nvidia/nemotron-3.5-lightning-30b-a3b[17:54:20] [INFO]   |-- tokens: input=329, output=1128, total=1457, tps=19[17:54:20] [INFO]   |-- requests: success=2, failed=2, total=4, rpm=3[17:54:20] [INFO] 📐 Measuring dataset column statistics:[17:54:20] [INFO]   |-- 🎲 column: 'patient_sampler'[17:54:20] [INFO]   |-- 🎲 column: 'doctor_sampler'[17:54:20] [INFO]   |-- 🎲 column: 'patient_id'[17:54:20] [INFO]   |-- 🧩 column: 'first_name'[17:54:20] [INFO]   |-- 🧩 column: 'last_name'[17:54:20] [INFO]   |-- 🧩 column: 'dob'[17:54:20] [INFO]   |-- 🎲 column: 'symptom_onset_date'[17:54:20] [INFO]   |-- 🎲 column: 'date_of_visit'[17:54:20] [INFO]   |-- 🧩 column: 'physician'[17:54:20] [INFO]   |-- 📝 column: 'physician_notes'[17:54:21] [INFO]   |-- 🌱 column: 'diagnosis'[17:54:21] [INFO]   |-- 🌱 column: 'patient_summary'[17:54:21] [INFO] 🙌 Preview complete!

Python

1 # Run this cell multiple times to cycle through the 2 preview records.
2 preview.display_sample_record()
3

Output

[index: 0]
                                                                                                              
                                                 Seed Columns                                                 
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name            ┃ Value                                                                                    ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ diagnosis       │ cervical spondylosis                                                                     │
├─────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ patient_summary │ I've been having a lot of pain in my neck and back. I've also been having trouble with   │
│                 │ my balance and coordination. I've been coughing a lot and my limbs feel weak.            │
└─────────────────┴──────────────────────────────────────────────────────────────────────────────────────────┘
                                                                                                              
                                                                                                              
                                              Generated Columns                                               
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name               ┃ Value                                                                                 ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ patient_sampler    │ {                                                                                     │
│                    │     'uuid': 'a6e6e968-5f2c-48da-81cf-f7a5b0dfba39',                                   │
│                    │     'locale': 'en_US',                                                                │
│                    │     'first_name': 'Eric',                                                             │
│                    │     'last_name': 'Lane',                                                              │
│                    │     'middle_name': None,                                                              │
│                    │     'sex': 'Male',                                                                    │
│                    │     'street_number': '23760',                                                         │
│                    │     'street_name': 'Davis Cove',                                                      │
│                    │     'city': 'West Lindafurt',                                                         │
│                    │     'state': 'Idaho',                                                                 │
│                    │     'postcode': '62129',                                                              │
│                    │     'age': 53,                                                                        │
│                    │     'birth_date': '1973-05-15',                                                       │
│                    │     'country': 'Albania',                                                             │
│                    │     'marital_status': 'separated',                                                    │
│                    │     'education_level': 'associates',                                                  │
│                    │     'unit': '',                                                                       │
│                    │     'occupation': 'Engineer, maintenance (IT)',                                       │
│                    │     'phone_number': '+1-256-381-5091x451',                                            │
│                    │     'bachelors_field': 'no_degree'                                                    │
│                    │ }                                                                                     │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ doctor_sampler     │ {                                                                                     │
│                    │     'uuid': '7646912f-f811-4d14-9819-ba5a5668f0ad',                                   │
│                    │     'locale': 'en_US',                                                                │
│                    │     'first_name': 'Brittney',                                                         │
│                    │     'last_name': 'Carter',                                                            │
│                    │     'middle_name': None,                                                              │
│                    │     'sex': 'Female',                                                                  │
│                    │     'street_number': '84177',                                                         │
│                    │     'street_name': 'Wilson Way',                                                      │
│                    │     'city': 'Lake Jeffrey',                                                           │
│                    │     'state': 'Kansas',                                                                │
│                    │     'postcode': '70749',                                                              │
│                    │     'age': 99,                                                                        │
│                    │     'birth_date': '1927-02-11',                                                       │
│                    │     'country': 'Papua New Guinea',                                                    │
│                    │     'marital_status': 'widowed',                                                      │
│                    │     'education_level': 'secondary_education',                                         │
│                    │     'unit': '',                                                                       │
│                    │     'occupation': 'Housing manager/officer',                                          │
│                    │     'phone_number': '247-754-6618x796',                                               │
│                    │     'bachelors_field': 'no_degree'                                                    │
│                    │ }                                                                                     │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ patient_id         │ PT-8F3C18F8                                                                           │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ symptom_onset_date │ 2024-12-24T00:00:00                                                                   │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ date_of_visit      │ 2024-12-31T00:00:00                                                                   │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ physician_notes    │ **Patient:** Eric Lane                                                                │
│                    │ **DOB:** [Redacted]                                                                   │
│                    │ **Date:** 2024-12-31                                                                  │
│                    │ **Chief Complaint:** Acute exacerbation of chronic cervical spondylosis with          │
│                    │ new-onset neurological symptoms and respiratory complaints.                           │
│                    │                                                                                       │
│                    │ **HPI:** Patient reports a significant decline in condition since initial diagnosis   │
│                    │ in late 2024. Primary complaint is severe, persistent neck and back pain, described   │
│                    │ as constant and debilitating. Patient specifically reports new and alarming symptoms: │
│                    │ progressive imbalance and significant coordination difficulties. Additionally,        │
│                    │ patient complains of generalized limb weakness and a persistent, productive cough.    │
│                    │ Denies recent trauma, but notes symptoms have been steadily worsening over the past   │
│                    │ week.                                                                                 │
│                    │                                                                                       │
│                    │ **ROS:**                                                                              │
│                    │ *   **Neuromuscular:** Patient affirms "drop attacks" feeling and weakness in         │
│                    │ extremities. Gait is unsteady.                                                        │
│                    │ *   **Respiratory:** Chronic cough present, worsening today. No reported fever or     │
│                    │ chills, but significant discomfort.                                                   │
│                    │ *   **Musculoskeletal:** Severe cervical and thoracic stiffness.                      │
│                    │                                                                                       │
│                    │ **Physical Exam:**                                                                    │
│                    │ *   **Vitals:** BP 138/85, HR 78, RR 18, Temp 36.8°C, SpO2 96% RA.                    │
│                    │ *   **Neuro:** Cranial nerves intact. Muscle strength 4/5 in bilateral upper and      │
│                    │ lower extremities. Reflexes 2+ at knees, 1+ at ankles. Babinski sign negative.        │
│                    │ Romberg's test positive (significant balance impairment). Sensation intact to light   │
│                    │ touch.                                                                                │
│                    │ *   **Musculoskeletal:** Limited range of motion in cervical spine due to pain and    │
│                    │ spasm. Tenderness noted C5-T1. No malar rash or joint swelling.                       │
│                    │ *   **Lungs:** Clear to auscultation bilaterally. No wheezing or crackles.            │
│                    │                                                                                       │
│                    │ **Assessment:**                                                                       │
│                    │ 1.  **Cervical Spondylosis:** Acute exacerbation with myelopathic features (balance   │
│                    │ issues, coordination difficulties, lower extremity weakness).                         │
│                    │ 2.  **Upper Respiratory Infection/Cough:** Viral etiology vs. irritation from         │
│                    │ post-nasal drip/mechanical factors.                                                   │
│                    │ 3.  **Muscle Weakness:** Deconditioning vs. potential neurological compromise.        │
│                    │                                                                                       │
│                    │ **Plan:**                                                                             │
│                    │ *   Order MRI cervical spine with and without contrast to rule out cord               │
│                    │ compression/stenosis progression.                                                     │
│                    │ *   Order CBC with differential and CRP/ESR to inflammatory markers.                  │
│                    │ *   Prescribe a short course of oral Prednisone 20mg daily for 5 days to reduce       │
│                    │ neuro-inflammation.                                                                   │
│                    │ *   Prescribe Cyclobenzaprine 10mg HS for muscle spasm and to aid sleep.              │
│                    │ *   Recommend strict cervical collar usage for 2 weeks to immobilize and rest the     │
│                    │ spine.                                                                                │
│                    │ *   Prescribe Guaifenesin for cough expectoration.                                    │
│                    │ *   Physical Therapy referral for gentle mobilization once acute inflammation         │
│                    │ subsides.                                                                             │
│                    │ *   **Safety Note:** Patient advised to seek immediate ED care if acute onset of      │
│                    │ urinary retention, saddle anesthesia, or acute respiratory distress occurs.           │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ first_name         │ Eric                                                                                  │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ last_name          │ Lane                                                                                  │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ dob                │ 1973-05-15                                                                            │
├────────────────────┼───────────────────────────────────────────────────────────────────────────────────────┤
│ physician          │ Dr. Carter                                                                            │
└────────────────────┴───────────────────────────────────────────────────────────────────────────────────────┘
                                                                                                              

Python

1 # The preview dataset is available as a pandas DataFrame.
2 preview.dataset
3

Output

  
      
      diagnosis
      patient_summary
      patient_sampler
      doctor_sampler
      patient_id
      symptom_onset_date
      date_of_visit
      dob
      last_name
      physician
      first_name
      physician_notes
    

  
      0
      cervical spondylosis
      I've been having a lot of pain in my neck and ...
      {'uuid': 'a6e6e968-5f2c-48da-81cf-f7a5b0dfba39...
      {'uuid': '7646912f-f811-4d14-9819-ba5a5668f0ad...
      PT-8F3C18F8
      2024-12-24T00:00:00
      2024-12-31T00:00:00
      1973-05-15
      Lane
      Dr. Carter
      Eric
      **Patient:** Eric Lane  \n**DOB:** [Redacted] ...
    

      1
      impetigo
      I have a rash on my face that is getting worse...
      {'uuid': '51f10a19-3619-4f2d-b133-e80719aeb40d...
      {'uuid': '0df15d44-e996-4c1d-a007-aef0996c5005...
      PT-B697C658
      2024-07-23T00:00:00
      2024-08-17T00:00:00
      1980-11-27
      Gonzales
      Dr. Nelson
      Linda
      ---\\n[2024-08-17 09:15] ROOM IN: Linda Gonzal...
    

	diagnosis	patient_summary	patient_sampler	doctor_sampler	patient_id	symptom_onset_date	date_of_visit	dob	last_name	physician	first_name	physician_notes
0	cervical spondylosis	I've been having a lot of pain in my neck and ...	{'uuid': 'a6e6e968-5f2c-48da-81cf-f7a5b0dfba39...	{'uuid': '7646912f-f811-4d14-9819-ba5a5668f0ad...	PT-8F3C18F8	2024-12-24T00:00:00	2024-12-31T00:00:00	1973-05-15	Lane	Dr. Carter	Eric	Patient: Eric Lane \nDOB: [Redacted] ...
1	impetigo	I have a rash on my face that is getting worse...	{'uuid': '51f10a19-3619-4f2d-b133-e80719aeb40d...	{'uuid': '0df15d44-e996-4c1d-a007-aef0996c5005...	PT-B697C658	2024-07-23T00:00:00	2024-08-17T00:00:00	1980-11-27	Gonzales	Dr. Nelson	Linda	---\\n[2024-08-17 09:15] ROOM IN: Linda Gonzal...

📊 Analyze the generated data

Data Designer automatically generates a basic statistical analysis of the generated data.
This analysis is available via the analysis property of generation result objects.

Python

1 # Print the analysis as a table.
2 preview.analysis.to_report()
3

Output

──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 2                               │ 12                              │ 100.0%                                      │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                              🌱 Seed-Dataset Columns                                              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                          ┃               data type ┃                           number unique values ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ diagnosis                            │                  string │                                     2 (100.0%) │
├──────────────────────────────────────┼─────────────────────────┼────────────────────────────────────────────────┤
│ patient_summary                      │                  string │                                     2 (100.0%) │
└──────────────────────────────────────┴─────────────────────────┴────────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                   ┃       data type ┃             number unique values ┃               sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ patient_sampler               │            dict │                       2 (100.0%) │          person_from_faker │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ doctor_sampler                │            dict │                       2 (100.0%) │          person_from_faker │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ patient_id                    │          string │                       2 (100.0%) │                       uuid │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ symptom_onset_date            │          string │                       2 (100.0%) │                   datetime │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ date_of_visit                 │          string │                       2 (100.0%) │                  timedelta │
└───────────────────────────────┴─────────────────┴──────────────────────────────────┴────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                📝 LLM-Text Columns                                                
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ physician_notes       │        string │                 2 (100.0%) │     135.0 +/- 4.0 │        549.5 +/- 103.9 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
                                               🧩 Expression Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                    ┃                 data type ┃                               number unique values ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ first_name                     │                    string │                                         2 (100.0%) │
├────────────────────────────────┼───────────────────────────┼────────────────────────────────────────────────────┤
│ last_name                      │                    string │                                         2 (100.0%) │
├────────────────────────────────┼───────────────────────────┼────────────────────────────────────────────────────┤
│ dob                            │                    string │                                         2 (100.0%) │
├────────────────────────────────┼───────────────────────────┼────────────────────────────────────────────────────┤
│ physician                      │                    string │                                         2 (100.0%) │
└────────────────────────────────┴───────────────────────────┴────────────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

🆙 Scale up!

Happy with your preview data?
Use the create method to submit larger Data Designer generation jobs.

Python

1 results = data_designer.create(config_builder, num_records=10, dataset_name="tutorial-3")
2

Output

[17:54:21] [INFO] OpenTelemetry metrics available at http://127.0.0.1:9464/metrics[17:54:21] [INFO] 🎨 Creating Data Designer dataset[17:54:21] [INFO]   |-- 🔒 Jinja rendering engine: secure[17:54:21] [INFO] ✅ Validation passed[17:54:21] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph[17:54:21] [INFO] Skipping model health checks because DATA_DESIGNER_SKIP_MODEL_HEALTH_CHECKS=1[17:54:21] [INFO] ⚡ Using async task-queue builder[17:54:21] [INFO] 📝 llm-text model config for column 'physician_notes'[17:54:21] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'[17:54:21] [INFO]   |-- model alias: 'nemotron-lightning'[17:54:21] [INFO]   |-- model provider: 'nvidia'[17:54:21] [INFO]   |-- inference parameters:[17:54:21] [INFO]   |  |-- generation_type=chat-completion[17:54:21] [INFO]   |  |-- max_parallel_requests=4[17:54:21] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}[17:54:21] [INFO]   |  |-- temperature=1.00[17:54:21] [INFO]   |  |-- top_p=0.95[17:54:21] [INFO]   |  |-- max_tokens=2048[17:54:21] [INFO] ⚡️ Async generation: 1 column(s) (column 'physician_notes'), 10 tasks across 1 row group(s)[17:54:21] [INFO] 🚀 (1/1) Dispatching with 10 records[17:54:21] [INFO] 🎲 (1/1) Preparing samplers to generate 10 records across 5 columns[17:54:21] [INFO] 🧩 (1/1) Generating column `dob` from expression[17:54:21] [INFO] 🧩 (1/1) Generating column `first_name` from expression[17:54:21] [INFO] 🧩 (1/1) Generating column `last_name` from expression[17:54:21] [INFO] 🧩 (1/1) Generating column `physician` from expression[17:54:21] [INFO] 🌱 (1/1) Sampling 10 records from seed dataset[17:54:21] [INFO]   |-- seed dataset size: 820 records[17:54:21] [INFO]   |-- sampling strategy: ordered[17:54:31] [INFO] 📊 Progress [10.0s]:[17:54:31] [INFO]   |-- 🌑 column 'physician_notes': 1/10 (10%) 0.1 rec/s[17:54:50] [INFO] 📊 Progress [28.9s]:[17:54:50] [INFO]   |-- 🌑 column 'physician_notes': 2/10 (20%) 0.1 rec/s[17:55:05] [INFO] 📊 Progress [44.4s]:[17:55:05] [INFO]   |-- 🌘 column 'physician_notes': 3/10 (30%) 0.1 rec/s[17:55:21] [WARNING] Observed retryable model-task error: kind=timeout; the row task will be deferred.[17:55:26] [INFO] 📊 Progress [65.7s]:[17:55:26] [INFO]   |-- 🌗 column 'physician_notes': 5/10 (50%) 0.1 rec/s[17:55:34] [INFO] 📊 Progress [72.9s]:[17:55:34] [INFO]   |-- 🌗 column 'physician_notes': 6/10 (60%) 0.1 rec/s[17:56:21] [INFO] 🔄 (1/1) Salvaging 4 deferred task(s)[17:56:32] [INFO] 📊 Progress [131.0s]:[17:56:32] [INFO]   |-- 🌗 column 'physician_notes': 7/10 (70%) 0.1 rec/s[17:56:40] [INFO] 📊 Progress [139.6s]:[17:56:40] [INFO]   |-- 🌖 column 'physician_notes': 9/10 (90%) 0.1 rec/s[17:58:21] [INFO] 📊 Progress [240.4s]:[17:58:21] [INFO]   |-- 🌕 column 'physician_notes': 10/10 (100%) 0.0 rec/s[17:58:21] [INFO] ✅ Async generation complete [240.5s]: 9 ok, 1 failed across 1 column(s)[17:58:21] [WARNING] ⚠️ Generated 9 of 10 requested records (90%). The dataset may be incomplete due to dropped rows.[17:58:21] [INFO] 📊 Model usage summary:[17:58:21] [INFO]   |-- model: nvidia/nemotron-3.5-lightning-30b-a3b[17:58:21] [INFO]   |-- tokens: input=1467, output=3576, total=5043, tps=20[17:58:21] [INFO]   |-- requests: success=9, failed=6, total=15, rpm=3[17:58:21] [INFO] 📐 Measuring dataset column statistics:[17:58:21] [INFO]   |-- 🎲 column: 'patient_sampler'[17:58:21] [INFO]   |-- 🎲 column: 'doctor_sampler'[17:58:21] [INFO]   |-- 🎲 column: 'patient_id'[17:58:21] [INFO]   |-- 🧩 column: 'first_name'[17:58:21] [INFO]   |-- 🧩 column: 'last_name'[17:58:21] [INFO]   |-- 🧩 column: 'dob'[17:58:21] [INFO]   |-- 🎲 column: 'symptom_onset_date'[17:58:21] [INFO]   |-- 🎲 column: 'date_of_visit'[17:58:21] [INFO]   |-- 🧩 column: 'physician'[17:58:21] [INFO]   |-- 📝 column: 'physician_notes'[17:58:21] [INFO]   |-- 🌱 column: 'diagnosis'[17:58:21] [INFO]   |-- 🌱 column: 'patient_summary'

Python

1 # Load the generated dataset as a pandas DataFrame.
2 dataset = results.load_dataset()
3  
4 dataset.head()
5

Output

  
      
      patient_sampler
      doctor_sampler
      patient_id
      symptom_onset_date
      date_of_visit
      dob
      first_name
      last_name
      physician
      diagnosis
      patient_summary
      physician_notes
    

  
      0
      {'uuid': '8111eb0e-d406-4607-88dd-9d0fe4d2a5ae...
      {'uuid': '4c253206-e3dd-494e-ad28-9c0ec67817d2...
      PT-00D8F510
      2024-01-06T00:00:00
      2024-01-09T00:00:00
      1938-11-29
      Kimberly
      Rojas
      Dr. Rice
      cervical spondylosis
      I've been having a lot of pain in my neck and ...
      **Patient:** Kimberly Rojas  
**Date of Visit:...
    

      1
      {'uuid': '1da9b4ae-59c5-4bfe-ae06-e7f646ff3f30...
      {'uuid': 'c6a5b37b-ef57-485a-a633-530f67db5099...
      PT-C42038BC
      2024-07-13T00:00:00
      2024-07-21T00:00:00
      2003-08-14
      Christina
      Dominguez
      Dr. Mosley
      impetigo
      I have a rash on my face that is getting worse...
      **Patient Name:** Christina Dominguez  
**Date...
    

      2
      {'uuid': 'e14b485b-9b37-4628-a34a-d451480f3da7...
      {'uuid': 'fd76d327-a4ff-4148-81bd-cc0441311839...
      PT-B6497204
      2024-05-29T00:00:00
      2024-06-04T00:00:00
      1983-04-25
      Breanna
      Jimenez
      Dr. Miller
      urinary tract infection
      I have been urinating blood. I sometimes feel ...
      **Patient Name:** Breanna Jimenez  
**Date of ...
    

      3
      {'uuid': 'd574a680-62dd-482a-a51b-a452d62bda5f...
      {'uuid': 'bfd0c19f-78d6-49af-a296-d4b6cf41f2d3...
      PT-4E397AC6
      2024-03-05T00:00:00
      2024-03-26T00:00:00
      1944-07-09
      Luis
      Perez
      Dr. Garcia
      arthritis
      I have been having trouble with my muscles and...
      **Patient Name:** Luis Perez  
**Date of Visit...
    

      4
      {'uuid': '5612365a-3cb0-4dca-b0ba-7b88f778337f...
      {'uuid': 'fceed474-d7aa-4a91-8fbb-e906f331a8bf...
      PT-9B6A9B39
      2024-03-30T00:00:00
      2024-04-11T00:00:00
      2005-03-21
      Jason
      Snyder
      Dr. Foster
      dengue
      I have been feeling really sick. My body hurts...
      **PATIENT NAME:** Jason Snyder  
**DATE OF APP...
    

	patient_sampler	doctor_sampler	patient_id	symptom_onset_date	date_of_visit	dob	first_name	last_name	physician	diagnosis	patient_summary	physician_notes
0	{'uuid': '8111eb0e-d406-4607-88dd-9d0fe4d2a5ae...	{'uuid': '4c253206-e3dd-494e-ad28-9c0ec67817d2...	PT-00D8F510	2024-01-06T00:00:00	2024-01-09T00:00:00	1938-11-29	Kimberly	Rojas	Dr. Rice	cervical spondylosis	I've been having a lot of pain in my neck and ...	Patient: Kimberly Rojas **Date of Visit:...
1	{'uuid': '1da9b4ae-59c5-4bfe-ae06-e7f646ff3f30...	{'uuid': 'c6a5b37b-ef57-485a-a633-530f67db5099...	PT-C42038BC	2024-07-13T00:00:00	2024-07-21T00:00:00	2003-08-14	Christina	Dominguez	Dr. Mosley	impetigo	I have a rash on my face that is getting worse...	Patient Name: Christina Dominguez **Date...
2	{'uuid': 'e14b485b-9b37-4628-a34a-d451480f3da7...	{'uuid': 'fd76d327-a4ff-4148-81bd-cc0441311839...	PT-B6497204	2024-05-29T00:00:00	2024-06-04T00:00:00	1983-04-25	Breanna	Jimenez	Dr. Miller	urinary tract infection	I have been urinating blood. I sometimes feel ...	Patient Name: Breanna Jimenez **Date of ...
3	{'uuid': 'd574a680-62dd-482a-a51b-a452d62bda5f...	{'uuid': 'bfd0c19f-78d6-49af-a296-d4b6cf41f2d3...	PT-4E397AC6	2024-03-05T00:00:00	2024-03-26T00:00:00	1944-07-09	Luis	Perez	Dr. Garcia	arthritis	I have been having trouble with my muscles and...	Patient Name: Luis Perez **Date of Visit...
4	{'uuid': '5612365a-3cb0-4dca-b0ba-7b88f778337f...	{'uuid': 'fceed474-d7aa-4a91-8fbb-e906f331a8bf...	PT-9B6A9B39	2024-03-30T00:00:00	2024-04-11T00:00:00	2005-03-21	Jason	Snyder	Dr. Foster	dengue	I have been feeling really sick. My body hurts...	PATIENT NAME: Jason Snyder **DATE OF APP...

Python

1 # Load the analysis results into memory.
2 analysis = results.load_analysis()
3  
4 analysis.to_report()
5

Output

──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 9                               │ 12                              │ 90.0%                                       │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                              🌱 Seed-Dataset Columns                                              
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                          ┃               data type ┃                           number unique values ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ diagnosis                            │                  string │                                      7 (77.8%) │
├──────────────────────────────────────┼─────────────────────────┼────────────────────────────────────────────────┤
│ patient_summary                      │                  string │                                     9 (100.0%) │
└──────────────────────────────────────┴─────────────────────────┴────────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                   ┃       data type ┃             number unique values ┃               sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ patient_sampler               │            dict │                       9 (100.0%) │          person_from_faker │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ doctor_sampler                │            dict │                       9 (100.0%) │          person_from_faker │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ patient_id                    │          string │                       9 (100.0%) │                       uuid │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ symptom_onset_date            │          string │                       9 (100.0%) │                   datetime │
├───────────────────────────────┼─────────────────┼──────────────────────────────────┼────────────────────────────┤
│ date_of_visit                 │          string │                       9 (100.0%) │                  timedelta │
└───────────────────────────────┴─────────────────┴──────────────────────────────────┴────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                📝 LLM-Text Columns                                                
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ physician_notes       │        string │                 9 (100.0%) │     132.0 +/- 5.0 │        390.0 +/- 172.1 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
                                               🧩 Expression Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                    ┃                 data type ┃                               number unique values ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ first_name                     │                    string │                                         9 (100.0%) │
├────────────────────────────────┼───────────────────────────┼────────────────────────────────────────────────────┤
│ last_name                      │                    string │                                         9 (100.0%) │
├────────────────────────────────┼───────────────────────────┼────────────────────────────────────────────────────┤
│ dob                            │                    string │                                         9 (100.0%) │
├────────────────────────────────┼───────────────────────────┼────────────────────────────────────────────────────┤
│ physician                      │                    string │                                         9 (100.0%) │
└────────────────────────────────┴───────────────────────────┴────────────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

⏭️ Next Steps

Check out the following notebook to learn more about: