Tutorials

The Basics

View as Markdown
Run in Google Colab

🎨 Data Designer Tutorial: The Basics

📚 What you'll learn

This notebook demonstrates the basics of Data Designer by generating a simple product review dataset.

📦 Import Data Designer

  • data_designer.config provides access to the configuration API.

  • DataDesigner is the main interface for data generation.

Python
1import data_designer.config as dd
2from data_designer.interface import DataDesigner
3

⚙️ Initialize the Data Designer interface

  • DataDesigner is the main object responsible for managing the data generation process.

  • When initialized without arguments, the default model providers are used.

Python
1data_designer = DataDesigner()
2

🎛️ Define model configurations

  • Each ModelConfig defines a model that can be used during the generation process.

  • The "model alias" is used to reference the model in the Data Designer config (as we will see below).

  • The "model provider" is the external service that hosts the model (see the model config docs for more details).

  • By default, we use build.nvidia.com as the model provider.

Python
1# This name is set in the model provider configuration.
2MODEL_PROVIDER = "nvidia"
3 
4# The model ID is from build.nvidia.com.
5MODEL_ID = "nvidia/nemotron-3.5-lightning-30b-a3b"
6 
7# We choose this alias to be descriptive for our use case.
8MODEL_ALIAS = "nemotron-lightning"
9 
10model_configs = [
11    dd.ModelConfig(
12        alias=MODEL_ALIAS,
13        model=MODEL_ID,
14        provider=MODEL_PROVIDER,
15        inference_parameters=dd.ChatCompletionInferenceParams(
16            temperature=1.0,
17            top_p=0.95,
18            max_tokens=2048,
19            extra_body={"chat_template_kwargs": {"enable_thinking": False}},
20        ),
21    )
22]
23

🏗️ Initialize the Data Designer Config Builder

  • The Data Designer config defines the dataset schema and generation process.

  • The config builder provides an intuitive interface for building this configuration.

  • The list of model configs is provided to the builder at initialization.

Python
1config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
2

🎲 Getting started with sampler columns

  • Sampler columns offer non-LLM based generation of synthetic data.

  • They are particularly useful for steering the diversity of the generated data, as we demonstrate below.


You can view available samplers using the config builder's info property:

Python
1config_builder.info.display("samplers")
2
Output
─────────────────────────────────────────── NeMo Data Designer Samplers ───────────────────────────────────────────

┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Type               ┃ Parameter                ┃ Data Type                         ┃ Required ┃ Constraints      ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ bernoulli          │ p                        │ number                            │    ✓     │ >= 0.0, <= 1.0   │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ bernoulli_mixture  │ p                        │ number                            │    ✓     │ >= 0.0, <= 1.0   │
│                    │ dist_name                │ string                            │    ✓     │                  │
│                    │ dist_params              │ dict                              │    ✓     │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ binomial           │ n                        │ integer                           │    ✓     │                  │
│                    │ p                        │ number                            │    ✓     │ >= 0.0, <= 1.0   │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ category           │ values                   │ string[] | integer[] | number[]   │    ✓     │ len > 1          │
│                    │ weights                  │ number[] | null                   │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ datetime           │ start                    │ string                            │    ✓     │                  │
│                    │ end                      │ string                            │    ✓     │                  │
│                    │ unit                     │ string                            │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ gaussian           │ mean                     │ number                            │    ✓     │                  │
│                    │ stddev                   │ number                            │    ✓     │                  │
│                    │ decimal_places           │ integer | null                    │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ person             │ locale                   │ string                            │          │                  │
│                    │ sex                      │ string | null                     │          │                  │
│                    │ city                     │ string | string[] | null          │          │                  │
│                    │ age_range                │ integer[]                         │          │ len > 2, len < 2 │
│                    │ select_field_values      │ object | null                     │          │                  │
│                    │ with_synthetic_personas  │ boolean                           │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ person_from_faker  │ locale                   │ string                            │          │                  │
│                    │ sex                      │ string | null                     │          │                  │
│                    │ city                     │ string | string[] | null          │          │                  │
│                    │ age_range                │ integer[]                         │          │ len > 2, len < 2 │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ poisson            │ mean                     │ number                            │    ✓     │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ scipy              │ dist_name                │ string                            │    ✓     │                  │
│                    │ dist_params              │ dict                              │    ✓     │                  │
│                    │ decimal_places           │ integer | null                    │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ subcategory        │ category                 │ string                            │    ✓     │                  │
│                    │ values                   │ dict                              │    ✓     │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ timedelta          │ dt_min                   │ integer                           │    ✓     │ >= 0             │
│                    │ dt_max                   │ integer                           │    ✓     │ > 0              │
│                    │ reference_column_name    │ string                            │    ✓     │                  │
│                    │ unit                     │ string                            │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ uniform            │ low                      │ number                            │    ✓     │                  │
│                    │ high                     │ number                            │    ✓     │                  │
│                    │ decimal_places           │ integer | null                    │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ uuid               │ prefix                   │ string | null                     │          │                  │
│                    │ short_form               │ boolean                           │          │                  │
│                    │ uppercase                │ boolean                           │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
└────────────────────┴──────────────────────────┴───────────────────────────────────┴──────────┴──────────────────┘

Let's start designing our product review dataset by adding product category and subcategory columns.

Python
1config_builder.add_column(
2    dd.SamplerColumnConfig(
3        name="product_category",
4        sampler_type=dd.SamplerType.CATEGORY,
5        params=dd.CategorySamplerParams(
6            values=[
7                "Electronics",
8                "Clothing",
9                "Home & Kitchen",
10                "Books",
11                "Home Office",
12            ],
13        ),
14    )
15)
16 
17config_builder.add_column(
18    dd.SamplerColumnConfig(
19        name="product_subcategory",
20        sampler_type=dd.SamplerType.SUBCATEGORY,
21        params=dd.SubcategorySamplerParams(
22            category="product_category",
23            values={
24                "Electronics": [
25                    "Smartphones",
26                    "Laptops",
27                    "Headphones",
28                    "Cameras",
29                    "Accessories",
30                ],
31                "Clothing": [
32                    "Men's Clothing",
33                    "Women's Clothing",
34                    "Winter Coats",
35                    "Activewear",
36                    "Accessories",
37                ],
38                "Home & Kitchen": [
39                    "Appliances",
40                    "Cookware",
41                    "Furniture",
42                    "Decor",
43                    "Organization",
44                ],
45                "Books": [
46                    "Fiction",
47                    "Non-Fiction",
48                    "Self-Help",
49                    "Textbooks",
50                    "Classics",
51                ],
52                "Home Office": [
53                    "Desks",
54                    "Chairs",
55                    "Storage",
56                    "Office Supplies",
57                    "Lighting",
58                ],
59            },
60        ),
61    )
62)
63 
64config_builder.add_column(
65    dd.SamplerColumnConfig(
66        name="target_age_range",
67        sampler_type=dd.SamplerType.CATEGORY,
68        params=dd.CategorySamplerParams(values=["18-25", "25-35", "35-50", "50-65", "65+"]),
69    )
70)
71 
72# Optionally validate that the columns are configured correctly.
73data_designer.validate(config_builder)
74
Output
[17:35:17] [INFO] ✅ Validation passed

Next, let's add samplers to generate data related to the customer and their review.

Python
1config_builder.add_column(
2    dd.SamplerColumnConfig(
3        name="customer",
4        sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
5        params=dd.PersonFromFakerSamplerParams(age_range=[18, 70], locale="en_US"),
6    )
7)
8 
9config_builder.add_column(
10    dd.SamplerColumnConfig(
11        name="number_of_stars",
12        sampler_type=dd.SamplerType.UNIFORM,
13        params=dd.UniformSamplerParams(low=1, high=5),
14        convert_to="int",  # Convert the sampled float to an integer.
15    )
16)
17 
18config_builder.add_column(
19    dd.SamplerColumnConfig(
20        name="review_style",
21        sampler_type=dd.SamplerType.CATEGORY,
22        params=dd.CategorySamplerParams(
23            values=["rambling", "brief", "detailed", "structured with bullet points"],
24            weights=[1, 2, 2, 1],
25        ),
26    )
27)
28 
29data_designer.validate(config_builder)
30
Output
[17:35:17] [INFO] ✅ Validation passed

🦜 LLM-generated columns

  • The real power of Data Designer comes from leveraging LLMs to generate text, code, and structured data.

  • When prompting the LLM, we can use Jinja templating to reference other columns in the dataset.

  • As we see below, nested json fields can be accessed using dot notation.

Python
1config_builder.add_column(
2    dd.LLMTextColumnConfig(
3        name="product_name",
4        prompt=(
5            "You are a helpful assistant that generates product names. DO NOT add quotes around the product name.\n\n"
6            "Come up with a creative product name for a product in the '{{ product_category }}' category, focusing "
7            "on products related to '{{ product_subcategory }}'. The target age range of the ideal customer is "
8            "{{ target_age_range }} years old. Respond with only the product name, no other text."
9        ),
10        model_alias=MODEL_ALIAS,
11    )
12)
13 
14config_builder.add_column(
15    dd.LLMTextColumnConfig(
16        name="customer_review",
17        prompt=(
18            "You are a customer named {{ customer.first_name }} from {{ customer.city }}, {{ customer.state }}. "
19            "You are {{ customer.age }} years old and recently purchased a product called {{ product_name }}. "
20            "Write a review of this product, which you gave a rating of {{ number_of_stars }} stars. "
21            "The style of the review should be '{{ review_style }}'. "
22            "Respond with only the review, no other text."
23        ),
24        model_alias=MODEL_ALIAS,
25    )
26)
27 
28data_designer.validate(config_builder)
29
Output
[17:35:17] [INFO] ✅ Validation passed

🔁 Iteration is key – preview the dataset!

  1. Use the preview method to generate a sample of records quickly.

  2. Inspect the results for quality and format issues.

  3. Adjust column configurations, prompts, or parameters as needed.

  4. Re-run the preview until satisfied.

Python
1preview = data_designer.preview(config_builder, num_records=2)
2
Output
[17:35:17] [INFO] 🕵️ Preview generation in progress
[17:35:17] [INFO]   |-- 🔒 Jinja rendering engine: secure
[17:35:17] [INFO] ✅ Validation passed
[17:35:17] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[17:35:17] [INFO] Skipping model health checks because DATA_DESIGNER_SKIP_MODEL_HEALTH_CHECKS=1
[17:35:18] [INFO] ⚡ Using async task-queue preview
[17:35:18] [INFO] 📝 llm-text model config for column 'product_name'
[17:35:18] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[17:35:18] [INFO]   |-- model alias: 'nemotron-lightning'
[17:35:18] [INFO]   |-- model provider: 'nvidia'
[17:35:18] [INFO]   |-- inference parameters:
[17:35:18] [INFO]   |  |-- generation_type=chat-completion
[17:35:18] [INFO]   |  |-- max_parallel_requests=4
[17:35:18] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[17:35:18] [INFO]   |  |-- temperature=1.00
[17:35:18] [INFO]   |  |-- top_p=0.95
[17:35:18] [INFO]   |  |-- max_tokens=2048
[17:35:18] [INFO] 📝 llm-text model config for column 'customer_review'
[17:35:18] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[17:35:18] [INFO]   |-- model alias: 'nemotron-lightning'
[17:35:18] [INFO]   |-- model provider: 'nvidia'
[17:35:18] [INFO]   |-- inference parameters:
[17:35:18] [INFO]   |  |-- generation_type=chat-completion
[17:35:18] [INFO]   |  |-- max_parallel_requests=4
[17:35:18] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[17:35:18] [INFO]   |  |-- temperature=1.00
[17:35:18] [INFO]   |  |-- top_p=0.95
[17:35:18] [INFO]   |  |-- max_tokens=2048
[17:35:18] [INFO] ⚡️ Async generation: 2 column(s) (column 'product_name', column 'customer_review'), 4 tasks across 1 row group(s)
[17:35:18] [INFO] 🚀 (1/1) Dispatching with 2 records
[17:35:18] [INFO] 🎲 (1/1) Preparing samplers to generate 2 records across 6 columns
[17:36:20] [WARNING] Observed retryable model-task error: kind=timeout; the row task will be deferred.
[17:36:20] [INFO] 🔄 (1/1) Salvaging 2 deferred task(s)
[17:36:20] [INFO] 📊 Progress [62.5s]:
[17:36:20] [INFO]   |-- 🚗 column 'product_name': 1/2 (50%) 0.0 rec/s
[17:36:20] [INFO]   |-- 🐱 column 'customer_review': 0/2 (0%) 0.0 rec/s
[17:36:26] [INFO] 📊 Progress [68.2s]:
[17:36:26] [INFO]   |-- 🚗 column 'product_name': 1/2 (50%) 0.0 rec/s
[17:36:26] [INFO]   |-- 😸 column 'customer_review': 1/2 (50%) 0.0 rec/s
[17:37:03] [INFO] 📊 Progress [105.6s]:
[17:37:03] [INFO]   |-- 🚀 column 'product_name': 2/2 (100%) 0.0 rec/s
[17:37:03] [INFO]   |-- 😸 column 'customer_review': 1/2 (50%) 0.0 rec/s
[17:37:11] [INFO] 📊 Progress [113.9s]:
[17:37:11] [INFO]   |-- 🚀 column 'product_name': 2/2 (100%) 0.0 rec/s
[17:37:11] [INFO]   |-- 🦁 column 'customer_review': 2/2 (100%) 0.0 rec/s
[17:37:11] [INFO] ✅ Async generation complete [113.9s]: 4 ok, 0 failed across 2 column(s)
[17:37:11] [INFO] 📊 Model usage summary:
[17:37:11] [INFO]   |-- model: nvidia/nemotron-3.5-lightning-30b-a3b
[17:37:11] [INFO]   |-- tokens: input=368, output=154, total=522, tps=4
[17:37:11] [INFO]   |-- requests: success=4, failed=2, total=6, rpm=3
[17:37:11] [INFO] 📐 Measuring dataset column statistics:
[17:37:11] [INFO]   |-- 🎲 column: 'product_category'
[17:37:11] [INFO]   |-- 🎲 column: 'product_subcategory'
[17:37:11] [INFO]   |-- 🎲 column: 'target_age_range'
[17:37:11] [INFO]   |-- 🎲 column: 'customer'
[17:37:11] [INFO]   |-- 🎲 column: 'number_of_stars'
[17:37:11] [INFO]   |-- 🎲 column: 'review_style'
[17:37:11] [INFO]   |-- 📝 column: 'product_name'
[17:37:12] [INFO]   |-- 📝 column: 'customer_review'
[17:37:12] [INFO] 🎊 Preview complete!
Python
1# Run this cell multiple times to cycle through the 2 preview records.
2preview.display_sample_record()
3
Output
[index: 0]
                                                                                                              
                                              Generated Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                ┃ Value                                                                                ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category    │ Home Office                                                                          │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ product_subcategory │ Chairs                                                                               │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ target_age_range    │ 50-65                                                                                │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ customer            │ {                                                                                    │
│                     │     'uuid': '5e24aa64-a5cd-4900-9dce-8ba012dce00d',                                  │
│                     │     'locale': 'en_US',                                                               │
│                     │     'first_name': 'Trevor',                                                          │
│                     │     'last_name': 'Johnston',                                                         │
│                     │     'middle_name': None,                                                             │
│                     │     'sex': 'Male',                                                                   │
│                     │     'street_number': '0485',                                                         │
│                     │     'street_name': 'Bradford Road',                                                  │
│                     │     'city': 'East Josephberg',                                                       │
│                     │     'state': 'Wyoming',                                                              │
│                     │     'postcode': '35428',                                                             │
│                     │     'age': 25,                                                                       │
│                     │     'birth_date': '2001-07-18',                                                      │
│                     │     'country': 'Switzerland',                                                        │
│                     │     'marital_status': 'never_married',                                               │
│                     │     'education_level': 'graduate',                                                   │
│                     │     'unit': '',                                                                      │
│                     │     'occupation': 'Embryologist, clinical',                                          │
│                     │     'phone_number': '970.220.6373',                                                  │
│                     │     'bachelors_field': 'education'                                                   │
│                     │ }                                                                                    │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ number_of_stars     │ 3                                                                                    │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ review_style        │ brief                                                                                │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ product_name        │ The userTruncated: 1500 words enabled, 500 tokens                                    │
│                     │                                                                                      │
│                     │ awaited yet?                                                                         │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ customer_review     │ I was pretty excited to get this, but honestly it's just okay. For the price, I      │
│                     │ expected a bit more punch and smoother performance. It works fine for basic stuff,   │
│                     │ but don't expect anything groundbreaking. It does the job, but don't rave about it   │
│                     │ to your friends. It’s functional, just not amazing.                                  │
└─────────────────────┴──────────────────────────────────────────────────────────────────────────────────────┘
Python
1# The preview dataset is available as a pandas DataFrame.
2preview.dataset
3
Output
product_category product_subcategory target_age_range customer number_of_stars review_style product_name customer_review
0 Home Office Chairs 50-65 {'uuid': '5e24aa64-a5cd-4900-9dce-8ba012dce00d... 3 brief The userTruncated: 1500 words enabled, 500 tok... I was pretty excited to get this, but honestly...
1 Books Non-Fiction 25-35 {'uuid': '13be400f-d06d-4249-bdb7-65716cd28229... 4 brief Re:View Re:View was a game-changer for my reading rout...

📊 Analyze the generated data

  • Data Designer automatically generates a basic statistical analysis of the generated data.

  • This analysis is available via the analysis property of generation result objects.

Python
1# Print the analysis as a table.
2preview.analysis.to_report()
3
Output
──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 2                               │ 8                               │ 100.0%                                      │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                    ┃       data type ┃            number unique values ┃               sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category               │          string │                      2 (100.0%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ product_subcategory            │          string │                      2 (100.0%) │                subcategory │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ target_age_range               │          string │                      2 (100.0%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ customer                       │            dict │                      2 (100.0%) │          person_from_faker │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ number_of_stars                │             int │                      2 (100.0%) │                    uniform │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ review_style                   │          string │                       1 (50.0%) │                   category │
└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                📝 LLM-Text Columns                                                
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_name          │        string │                 2 (100.0%) │      74.0 +/- 0.0 │          11.0 +/- 11.3 │
├───────────────────────┼───────────────┼────────────────────────────┼───────────────────┼────────────────────────┤
│ customer_review       │        string │                 2 (100.0%) │      74.5 +/- 7.5 │           59.5 +/- 9.2 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

🆙 Scale up!

  • Happy with your preview data?

  • Use the create method to submit larger Data Designer generation jobs.

Python
1results = data_designer.create(config_builder, num_records=10, dataset_name="tutorial-1")
2
Output
[17:37:12] [INFO] OpenTelemetry metrics available at http://127.0.0.1:9464/metrics
[17:37:12] [INFO] 🎨 Creating Data Designer dataset
[17:37:12] [INFO]   |-- 🔒 Jinja rendering engine: secure
[17:37:12] [INFO] ✅ Validation passed
[17:37:12] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[17:37:12] [INFO] Skipping model health checks because DATA_DESIGNER_SKIP_MODEL_HEALTH_CHECKS=1
[17:37:12] [INFO] ⚡ Using async task-queue builder
[17:37:12] [INFO] 📝 llm-text model config for column 'product_name'
[17:37:12] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[17:37:12] [INFO]   |-- model alias: 'nemotron-lightning'
[17:37:12] [INFO]   |-- model provider: 'nvidia'
[17:37:12] [INFO]   |-- inference parameters:
[17:37:12] [INFO]   |  |-- generation_type=chat-completion
[17:37:12] [INFO]   |  |-- max_parallel_requests=4
[17:37:12] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[17:37:12] [INFO]   |  |-- temperature=1.00
[17:37:12] [INFO]   |  |-- top_p=0.95
[17:37:12] [INFO]   |  |-- max_tokens=2048
[17:37:12] [INFO] 📝 llm-text model config for column 'customer_review'
[17:37:12] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[17:37:12] [INFO]   |-- model alias: 'nemotron-lightning'
[17:37:12] [INFO]   |-- model provider: 'nvidia'
[17:37:12] [INFO]   |-- inference parameters:
[17:37:12] [INFO]   |  |-- generation_type=chat-completion
[17:37:12] [INFO]   |  |-- max_parallel_requests=4
[17:37:12] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[17:37:12] [INFO]   |  |-- temperature=1.00
[17:37:12] [INFO]   |  |-- top_p=0.95
[17:37:12] [INFO]   |  |-- max_tokens=2048
[17:37:12] [INFO] ⚡️ Async generation: 2 column(s) (column 'product_name', column 'customer_review'), 20 tasks across 1 row group(s)
[17:37:12] [INFO] 🚀 (1/1) Dispatching with 10 records
[17:37:12] [INFO] 🎲 (1/1) Preparing samplers to generate 10 records across 6 columns
[17:37:35] [INFO] 📊 Progress [22.6s]:
[17:37:35] [INFO]   |-- 🚶 column 'product_name': 1/10 (10%) 0.0 rec/s
[17:37:35] [INFO]   |-- 🚶 column 'customer_review': 1/10 (10%) 0.0 rec/s
[17:37:54] [INFO] 📊 Progress [41.8s]:
[17:37:54] [INFO]   |-- 🐴 column 'product_name': 3/10 (30%) 0.1 rec/s
[17:37:54] [INFO]   |-- 🚶 column 'customer_review': 1/10 (10%) 0.0 rec/s
[17:38:01] [INFO] 📊 Progress [48.4s]:
[17:38:01] [INFO]   |-- 🐴 column 'product_name': 4/10 (40%) 0.1 rec/s
[17:38:01] [INFO]   |-- 🚶 column 'customer_review': 2/10 (20%) 0.0 rec/s
[17:38:12] [WARNING] Observed retryable model-task error: kind=timeout; the row task will be deferred.
[17:38:15] [INFO] 📊 Progress [63.0s]:
[17:38:15] [INFO]   |-- 🚗 column 'product_name': 5/10 (50%) 0.1 rec/s
[17:38:15] [INFO]   |-- 🚶 column 'customer_review': 2/10 (20%) 0.0 rec/s
[17:38:51] [INFO] 📊 Progress [98.4s]:
[17:38:51] [INFO]   |-- 🚗 column 'product_name': 6/10 (60%) 0.1 rec/s
[17:38:51] [INFO]   |-- 🐴 column 'customer_review': 3/10 (30%) 0.0 rec/s
[17:38:58] [INFO] 📊 Progress [105.3s]:
[17:38:58] [INFO]   |-- 🚗 column 'product_name': 7/10 (70%) 0.1 rec/s
[17:38:58] [INFO]   |-- 🐴 column 'customer_review': 3/10 (30%) 0.0 rec/s
[17:39:06] [INFO] 📊 Progress [113.5s]:
[17:39:06] [INFO]   |-- ✈️ column 'product_name': 8/10 (80%) 0.1 rec/s
[17:39:06] [INFO]   |-- 🐴 column 'customer_review': 3/10 (30%) 0.0 rec/s
[17:39:17] [INFO] 📊 Progress [125.1s]:
[17:39:17] [INFO]   |-- ✈️ column 'product_name': 8/10 (80%) 0.1 rec/s
[17:39:17] [INFO]   |-- 🐴 column 'customer_review': 4/10 (40%) 0.0 rec/s
[17:40:01] [INFO] 📊 Progress [168.6s]:
[17:40:01] [INFO]   |-- ✈️ column 'product_name': 8/10 (80%) 0.0 rec/s
[17:40:01] [INFO]   |-- 🚗 column 'customer_review': 5/10 (50%) 0.0 rec/s
[17:40:01] [INFO] 🔄 (1/1) Salvaging 5 deferred task(s)
[17:40:34] [INFO] 📊 Progress [201.5s]:
[17:40:34] [INFO]   |-- 🚀 column 'product_name': 10/10 (100%) 0.0 rec/s
[17:40:34] [INFO]   |-- 🚗 column 'customer_review': 5/10 (50%) 0.0 rec/s
[17:40:42] [INFO] 📊 Progress [209.7s]:
[17:40:42] [INFO]   |-- 🚀 column 'product_name': 10/10 (100%) 0.0 rec/s
[17:40:42] [INFO]   |-- 🚗 column 'customer_review': 6/10 (60%) 0.0 rec/s
[17:41:03] [INFO] 📊 Progress [230.7s]:
[17:41:03] [INFO]   |-- 🚀 column 'product_name': 10/10 (100%) 0.0 rec/s
[17:41:03] [INFO]   |-- 🚗 column 'customer_review': 7/10 (70%) 0.0 rec/s
[17:41:20] [INFO] 📊 Progress [247.8s]:
[17:41:20] [INFO]   |-- 🚀 column 'product_name': 10/10 (100%) 0.0 rec/s
[17:41:20] [INFO]   |-- ✈️ column 'customer_review': 8/10 (80%) 0.0 rec/s
[17:42:04] [INFO] 📊 Progress [292.0s]:
[17:42:04] [INFO]   |-- 🚀 column 'product_name': 10/10 (100%) 0.0 rec/s
[17:42:04] [INFO]   |-- ✈️ column 'customer_review': 9/10 (90%) 0.0 rec/s
[17:42:04] [INFO] 📊 Progress [292.0s]:
[17:42:04] [INFO]   |-- 🚀 column 'product_name': 10/10 (100%) 0.0 rec/s
[17:42:04] [INFO]   |-- 🚀 column 'customer_review': 10/10 (100%) 0.0 rec/s
[17:42:04] [INFO] ✅ Async generation complete [292.0s]: 18 ok, 2 failed across 2 column(s)
[17:42:05] [WARNING] ⚠️ Generated 8 of 10 requested records (80%). The dataset may be incomplete due to dropped rows.
[17:42:05] [INFO] 📊 Model usage summary:
[17:42:05] [INFO]   |-- model: nvidia/nemotron-3.5-lightning-30b-a3b
[17:42:05] [INFO]   |-- tokens: input=1592, output=2761, total=4353, tps=14
[17:42:05] [INFO]   |-- requests: success=18, failed=10, total=28, rpm=5
[17:42:05] [INFO] 📐 Measuring dataset column statistics:
[17:42:05] [INFO]   |-- 🎲 column: 'product_category'
[17:42:05] [INFO]   |-- 🎲 column: 'product_subcategory'
[17:42:05] [INFO]   |-- 🎲 column: 'target_age_range'
[17:42:05] [INFO]   |-- 🎲 column: 'customer'
[17:42:05] [INFO]   |-- 🎲 column: 'number_of_stars'
[17:42:05] [INFO]   |-- 🎲 column: 'review_style'
[17:42:05] [INFO]   |-- 📝 column: 'product_name'
[17:42:05] [INFO]   |-- 📝 column: 'customer_review'
Python
1# Load the generated dataset as a pandas DataFrame.
2dataset = results.load_dataset()
3 
4dataset.head()
5
Output
product_category product_subcategory target_age_range customer number_of_stars review_style product_name customer_review
0 Books Classics 65+ {'uuid': 'f3bcd6ee-02ee-4963-87cb-6a0c565ca119... 2 brief The Golden Page Purchased The Golden Page but found it underwh...
1 Books Fiction 35-50 {'uuid': '01277f4a-02d3-4cf8-ac51-e9f5f2715c38... 4 rambling The Artifact Hunter Okay so like I finally got The Artifact Hunter...
2 Electronics Smartphones 65+ {'uuid': 'f08bfae0-7c51-4f3f-8f93-dc12e6bbeb46... 4 structured with bullet points EasyLink Ultra * EasyLink Ultra arrived quickly and the packa...
3 Home Office Office Supplies 50-65 {'uuid': 'd24d1967-fb40-4681-b837-79a8b52931f5... 1 detailed MemoryKeeper Desk Organizer I honestly wanted to like the MemoryKeeper Des...
4 Home & Kitchen Decor 18-25 {'uuid': '59457f54-7b38-4630-b2cb-b8fe8024bc66... 1 detailed LumenShift Mirror I am writing this review with a heavy heart, a...
Python
1# Load the analysis results into memory.
2analysis = results.load_analysis()
3 
4analysis.to_report()
5
Output
──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 8                               │ 8                               │ 80.0%                                       │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                    ┃       data type ┃            number unique values ┃               sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category               │          string │                       5 (62.5%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ product_subcategory            │          string │                       7 (87.5%) │                subcategory │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ target_age_range               │          string │                       4 (50.0%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ customer                       │            dict │                      8 (100.0%) │          person_from_faker │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ number_of_stars                │             int │                       4 (50.0%) │                    uniform │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ review_style                   │          string │                       4 (50.0%) │                   category │
└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                📝 LLM-Text Columns                                                
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_name          │        string │                 8 (100.0%) │      73.5 +/- 0.7 │            4.0 +/- 1.5 │
├───────────────────────┼───────────────┼────────────────────────────┼───────────────────┼────────────────────────┤
│ customer_review       │        string │                 8 (100.0%) │      68.0 +/- 1.6 │        344.5 +/- 236.0 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

⏭️ Next Steps

Now that you've seen the basics of Data Designer, check out the following notebooks to learn more about: