Skip to navigation
Tutorials

The Basics

View as Markdown
Run in Google Colab

🎨 Data Designer Tutorial: The Basics

📚 What you'll learn

This notebook demonstrates the basics of Data Designer by generating a simple product review dataset.

📦 Import Data Designer

  • data_designer.config provides access to the configuration API.

  • DataDesigner is the main interface for data generation.

Python
1import data_designer.config as dd
2from data_designer.interface import DataDesigner
3

⚙️ Initialize the Data Designer interface

  • DataDesigner is the main object responsible for managing the data generation process.

  • When initialized without arguments, the default model providers are used.

Python
1data_designer = DataDesigner()
2

🎛️ Define model configurations

  • Each ModelConfig defines a model that can be used during the generation process.

  • The "model alias" is used to reference the model in the Data Designer config (as we will see below).

  • The "model provider" is the external service that hosts the model (see the model config docs for more details).

  • By default, we use build.nvidia.com as the model provider.

Python
1# This name is set in the model provider configuration.
2MODEL_PROVIDER = "nvidia"
3 
4# The model ID is from build.nvidia.com.
5MODEL_ID = "nvidia/nemotron-3.5-lightning-30b-a3b"
6 
7# We choose this alias to be descriptive for our use case.
8MODEL_ALIAS = "nemotron-lightning"
9 
10model_configs = [
11    dd.ModelConfig(
12        alias=MODEL_ALIAS,
13        model=MODEL_ID,
14        provider=MODEL_PROVIDER,
15        inference_parameters=dd.ChatCompletionInferenceParams(
16            temperature=1.0,
17            top_p=0.95,
18            max_tokens=2048,
19            extra_body={"chat_template_kwargs": {"enable_thinking": False}},
20        ),
21    )
22]
23

🏗️ Initialize the Data Designer Config Builder

  • The Data Designer config defines the dataset schema and generation process.

  • The config builder provides an intuitive interface for building this configuration.

  • The list of model configs is provided to the builder at initialization.

Python
1config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
2

🎲 Getting started with sampler columns

  • Sampler columns offer non-LLM based generation of synthetic data.

  • They are particularly useful for steering the diversity of the generated data, as we demonstrate below.


You can view available samplers using the config builder's info property:

Python
1config_builder.info.display("samplers")
2
Output
─────────────────────────────────────────── NeMo Data Designer Samplers ───────────────────────────────────────────

┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Type               ┃ Parameter                ┃ Data Type                         ┃ Required ┃ Constraints      ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ bernoulli          │ p                        │ number                            │    ✓     │ >= 0.0, <= 1.0   │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ bernoulli_mixture  │ p                        │ number                            │    ✓     │ >= 0.0, <= 1.0   │
│                    │ dist_name                │ string                            │    ✓     │                  │
│                    │ dist_params              │ dict                              │    ✓     │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ binomial           │ n                        │ integer                           │    ✓     │                  │
│                    │ p                        │ number                            │    ✓     │ >= 0.0, <= 1.0   │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ category           │ values                   │ string[] | integer[] | number[]   │    ✓     │ len > 1          │
│                    │ weights                  │ number[] | null                   │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ datetime           │ start                    │ string                            │    ✓     │                  │
│                    │ end                      │ string                            │    ✓     │                  │
│                    │ unit                     │ string                            │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ gaussian           │ mean                     │ number                            │    ✓     │                  │
│                    │ stddev                   │ number                            │    ✓     │                  │
│                    │ decimal_places           │ integer | null                    │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ person             │ locale                   │ string                            │          │                  │
│                    │ sex                      │ string | null                     │          │                  │
│                    │ city                     │ string | string[] | null          │          │                  │
│                    │ age_range                │ integer[]                         │          │ len > 2, len < 2 │
│                    │ select_field_values      │ object | null                     │          │                  │
│                    │ with_synthetic_personas  │ boolean                           │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ person_from_faker  │ locale                   │ string                            │          │                  │
│                    │ sex                      │ string | null                     │          │                  │
│                    │ city                     │ string | string[] | null          │          │                  │
│                    │ age_range                │ integer[]                         │          │ len > 2, len < 2 │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ poisson            │ mean                     │ number                            │    ✓     │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ scipy              │ dist_name                │ string                            │    ✓     │                  │
│                    │ dist_params              │ dict                              │    ✓     │                  │
│                    │ decimal_places           │ integer | null                    │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ subcategory        │ category                 │ string                            │    ✓     │                  │
│                    │ values                   │ dict                              │    ✓     │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ timedelta          │ dt_min                   │ integer                           │    ✓     │ >= 0             │
│                    │ dt_max                   │ integer                           │    ✓     │ > 0              │
│                    │ reference_column_name    │ string                            │    ✓     │                  │
│                    │ unit                     │ string                            │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ uniform            │ low                      │ number                            │    ✓     │                  │
│                    │ high                     │ number                            │    ✓     │                  │
│                    │ decimal_places           │ integer | null                    │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
├────────────────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────────────┤
│ uuid               │ prefix                   │ string | null                     │          │                  │
│                    │ short_form               │ boolean                           │          │                  │
│                    │ uppercase                │ boolean                           │          │                  │
│                    │ sampler_type             │ string                            │          │                  │
└────────────────────┴──────────────────────────┴───────────────────────────────────┴──────────┴──────────────────┘

Let's start designing our product review dataset by adding product category and subcategory columns.

Python
1config_builder.add_column(
2    dd.SamplerColumnConfig(
3        name="product_category",
4        sampler_type=dd.SamplerType.CATEGORY,
5        params=dd.CategorySamplerParams(
6            values=[
7                "Electronics",
8                "Clothing",
9                "Home & Kitchen",
10                "Books",
11                "Home Office",
12            ],
13        ),
14    )
15)
16 
17config_builder.add_column(
18    dd.SamplerColumnConfig(
19        name="product_subcategory",
20        sampler_type=dd.SamplerType.SUBCATEGORY,
21        params=dd.SubcategorySamplerParams(
22            category="product_category",
23            values={
24                "Electronics": [
25                    "Smartphones",
26                    "Laptops",
27                    "Headphones",
28                    "Cameras",
29                    "Accessories",
30                ],
31                "Clothing": [
32                    "Men's Clothing",
33                    "Women's Clothing",
34                    "Winter Coats",
35                    "Activewear",
36                    "Accessories",
37                ],
38                "Home & Kitchen": [
39                    "Appliances",
40                    "Cookware",
41                    "Furniture",
42                    "Decor",
43                    "Organization",
44                ],
45                "Books": [
46                    "Fiction",
47                    "Non-Fiction",
48                    "Self-Help",
49                    "Textbooks",
50                    "Classics",
51                ],
52                "Home Office": [
53                    "Desks",
54                    "Chairs",
55                    "Storage",
56                    "Office Supplies",
57                    "Lighting",
58                ],
59            },
60        ),
61    )
62)
63 
64config_builder.add_column(
65    dd.SamplerColumnConfig(
66        name="target_age_range",
67        sampler_type=dd.SamplerType.CATEGORY,
68        params=dd.CategorySamplerParams(values=["18-25", "25-35", "35-50", "50-65", "65+"]),
69    )
70)
71 
72# Optionally validate that the columns are configured correctly.
73data_designer.validate(config_builder)
74
Output
[02:04:02] [INFO] ✅ Validation passed

Next, let's add samplers to generate data related to the customer and their review.

Python
1config_builder.add_column(
2    dd.SamplerColumnConfig(
3        name="customer",
4        sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
5        params=dd.PersonFromFakerSamplerParams(age_range=[18, 70], locale="en_US"),
6    )
7)
8 
9config_builder.add_column(
10    dd.SamplerColumnConfig(
11        name="number_of_stars",
12        sampler_type=dd.SamplerType.UNIFORM,
13        params=dd.UniformSamplerParams(low=1, high=5),
14        convert_to="int",  # Convert the sampled float to an integer.
15    )
16)
17 
18config_builder.add_column(
19    dd.SamplerColumnConfig(
20        name="review_style",
21        sampler_type=dd.SamplerType.CATEGORY,
22        params=dd.CategorySamplerParams(
23            values=["rambling", "brief", "detailed", "structured with bullet points"],
24            weights=[1, 2, 2, 1],
25        ),
26    )
27)
28 
29data_designer.validate(config_builder)
30
Output
[02:04:02] [INFO] ✅ Validation passed

🦜 LLM-generated columns

  • The real power of Data Designer comes from leveraging LLMs to generate text, code, and structured data.

  • When prompting the LLM, we can use Jinja templating to reference other columns in the dataset.

  • As we see below, nested json fields can be accessed using dot notation.

Python
1config_builder.add_column(
2    dd.LLMTextColumnConfig(
3        name="product_name",
4        prompt=(
5            "You are a helpful assistant that generates product names. DO NOT add quotes around the product name.\n\n"
6            "Come up with a creative product name for a product in the '{{ product_category }}' category, focusing "
7            "on products related to '{{ product_subcategory }}'. The target age range of the ideal customer is "
8            "{{ target_age_range }} years old. Respond with only the product name, no other text."
9        ),
10        model_alias=MODEL_ALIAS,
11    )
12)
13 
14config_builder.add_column(
15    dd.LLMTextColumnConfig(
16        name="customer_review",
17        prompt=(
18            "You are a customer named {{ customer.first_name }} from {{ customer.city }}, {{ customer.state }}. "
19            "You are {{ customer.age }} years old and recently purchased a product called {{ product_name }}. "
20            "Write a review of this product, which you gave a rating of {{ number_of_stars }} stars. "
21            "The style of the review should be '{{ review_style }}'. "
22            "Respond with only the review, no other text."
23        ),
24        model_alias=MODEL_ALIAS,
25    )
26)
27 
28data_designer.validate(config_builder)
29
Output
[02:04:02] [INFO] ✅ Validation passed

🔁 Iteration is key – preview the dataset!

  1. Use the preview method to generate a sample of records quickly.

  2. Inspect the results for quality and format issues.

  3. Adjust column configurations, prompts, or parameters as needed.

  4. Re-run the preview until satisfied.

Python
1preview = data_designer.preview(config_builder, num_records=2)
2
Output
[02:04:02] [INFO] 🖼️ Preview generation in progress
[02:04:02] [INFO]   |-- 🔒 Jinja rendering engine: secure
[02:04:02] [INFO] ✅ Validation passed
[02:04:02] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[02:04:02] [INFO] Skipping model health checks because DATA_DESIGNER_SKIP_MODEL_HEALTH_CHECKS=1
[02:04:02] [INFO] ⚡ Using async task-queue preview
[02:04:02] [INFO] 📝 llm-text model config for column 'product_name'
[02:04:02] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[02:04:02] [INFO]   |-- model alias: 'nemotron-lightning'
[02:04:02] [INFO]   |-- model provider: 'nvidia'
[02:04:02] [INFO]   |-- inference parameters:
[02:04:02] [INFO]   |  |-- generation_type=chat-completion
[02:04:02] [INFO]   |  |-- max_parallel_requests=4
[02:04:02] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[02:04:02] [INFO]   |  |-- temperature=1.00
[02:04:02] [INFO]   |  |-- top_p=0.95
[02:04:02] [INFO]   |  |-- max_tokens=2048
[02:04:02] [INFO] 📝 llm-text model config for column 'customer_review'
[02:04:02] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[02:04:02] [INFO]   |-- model alias: 'nemotron-lightning'
[02:04:02] [INFO]   |-- model provider: 'nvidia'
[02:04:02] [INFO]   |-- inference parameters:
[02:04:02] [INFO]   |  |-- generation_type=chat-completion
[02:04:02] [INFO]   |  |-- max_parallel_requests=4
[02:04:02] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[02:04:02] [INFO]   |  |-- temperature=1.00
[02:04:02] [INFO]   |  |-- top_p=0.95
[02:04:02] [INFO]   |  |-- max_tokens=2048
[02:04:02] [INFO] ⚡️ Async generation: 2 column(s) (column 'product_name', column 'customer_review'), 4 tasks across 1 row group(s)
[02:04:02] [INFO] 🚀 (1/1) Dispatching with 2 records
[02:04:02] [INFO] 🎲 (1/1) Preparing samplers to generate 2 records across 6 columns
[02:04:06] [INFO] 📊 Progress [4.1s]:
[02:04:06] [INFO]   |-- 🚀 column 'product_name': 2/2 (100%) 0.5 rec/s
[02:04:06] [INFO]   |-- 🐔 column 'customer_review': 2/2 (100%) 0.5 rec/s
[02:04:06] [INFO] ✅ Async generation complete [4.1s]: 4 ok, 0 failed across 2 column(s)
[02:04:06] [INFO] 📊 Model usage summary:
[02:04:06] [INFO]   |-- model: nvidia/nemotron-3.5-lightning-30b-a3b
[02:04:06] [INFO]   |-- tokens: input=354, output=276, reasoning=0, total=630, tps=151
[02:04:06] [INFO]   |-- requests: success=4, failed=0, total=4, rpm=57
[02:04:06] [INFO] 📐 Measuring dataset column statistics:
[02:04:06] [INFO]   |-- 🎲 column: 'product_category'
[02:04:06] [INFO]   |-- 🎲 column: 'product_subcategory'
[02:04:06] [INFO]   |-- 🎲 column: 'target_age_range'
[02:04:06] [INFO]   |-- 🎲 column: 'customer'
[02:04:06] [INFO]   |-- 🎲 column: 'number_of_stars'
[02:04:06] [INFO]   |-- 🎲 column: 'review_style'
[02:04:06] [INFO]   |-- 📝 column: 'product_name'
[02:04:06] [INFO]   |-- 📝 column: 'customer_review'
[02:04:06] [INFO] 👏 Preview complete!
Python
1# Run this cell multiple times to cycle through the 2 preview records.
2preview.display_sample_record()
3
Output
[index: 0]
                                                                                                              
                                              Generated Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                ┃ Value                                                                                ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category    │ Home & Kitchen                                                                       │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ product_subcategory │ Furniture                                                                            │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ target_age_range    │ 50-65                                                                                │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ customer            │ {                                                                                    │
│                     │     'uuid': '55596e24-c42a-4fc7-b6c0-2bba17e997f5',                                  │
│                     │     'locale': 'en_US',                                                               │
│                     │     'first_name': 'Angela',                                                          │
│                     │     'last_name': 'Camacho',                                                          │
│                     │     'middle_name': None,                                                             │
│                     │     'sex': 'Female',                                                                 │
│                     │     'street_number': '14171',                                                        │
│                     │     'street_name': 'Ferrell Groves',                                                 │
│                     │     'city': 'Joymouth',                                                              │
│                     │     'state': 'New Mexico',                                                           │
│                     │     'postcode': '42936',                                                             │
│                     │     'age': 35,                                                                       │
│                     │     'birth_date': '1991-08-10',                                                      │
│                     │     'country': 'Nauru',                                                              │
│                     │     'marital_status': 'never_married',                                               │
│                     │     'education_level': 'graduate',                                                   │
│                     │     'unit': '',                                                                      │
│                     │     'occupation': 'Psychotherapist, child',                                          │
│                     │     'phone_number': '861.899.2113x89297',                                            │
│                     │     'bachelors_field': 'education'                                                   │
│                     │ }                                                                                    │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ number_of_stars     │ 1                                                                                    │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ review_style        │ rambling                                                                             │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ product_name        │ SupportiveArms Sofa                                                                  │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────┤
│ customer_review     │ Oh my gosh you guys I just have to get this off my chest because I am literally      │
│                     │ shaking right now I just opened the box and I am so excited but also so mad I paid   │
│                     │ four hundred dollars for this and it is literally the flimsiest thing I have ever    │
│                     │ felt in my life the arms are like made of like cardboard basically and when I sit    │
│                     │ down my legs just kind of splay out everywhere it’s so embarrassing for a sofa that  │
│                     │ cost this much money and the cushions are like rocks they don’t even squish I’ve had │
│                     │ it for like two days and already one of the pillows has a giant stain on it from the │
│                     │ cat I mean seriously what is wrong with quality control these days I tried to call   │
│                     │ customer service but the lady was so rude she just kept putting me on hold and then  │
│                     │ the line just dropped dead I am so over it I want my money back but I don’t even     │
│                     │ want to look at this thing ever again it’s just sitting in my living room looking    │
│                     │ like a giant lumpy rock I hate it I hate it I hate it lol                            │
└─────────────────────┴──────────────────────────────────────────────────────────────────────────────────────┘
Python
1# The preview dataset is available as a pandas DataFrame.
2preview.dataset
3
Output
product_category product_subcategory target_age_range customer number_of_stars review_style product_name customer_review
0 Home & Kitchen Furniture 50-65 {'uuid': '55596e24-c42a-4fc7-b6c0-2bba17e997f5... 1 rambling SupportiveArms Sofa Oh my gosh you guys I just have to get this of...
1 Home Office Lighting 18-25 {'uuid': 'fbad5275-88cf-4e36-956e-251b09de3949... 1 brief AuroraDesk Just finished setting this up and it's total g...

📊 Analyze the generated data

  • Data Designer automatically generates a basic statistical analysis of the generated data.

  • This analysis is available via the analysis property of generation result objects.

Python
1# Print the analysis as a table.
2preview.analysis.to_report()
3
Output
──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 2                               │ 8                               │ 100.0%                                      │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                    ┃       data type ┃            number unique values ┃               sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category               │          string │                      2 (100.0%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ product_subcategory            │          string │                      2 (100.0%) │                subcategory │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ target_age_range               │          string │                      2 (100.0%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ customer                       │            dict │                      2 (100.0%) │          person_from_faker │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ number_of_stars                │             int │                       1 (50.0%) │                    uniform │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ review_style                   │          string │                      2 (100.0%) │                   category │
└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                📝 LLM-Text Columns                                                
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_name          │        string │                 2 (100.0%) │      74.5 +/- 0.5 │            4.5 +/- 0.7 │
├───────────────────────┼───────────────┼────────────────────────────┼───────────────────┼────────────────────────┤
│ customer_review       │        string │                 2 (100.0%) │      68.0 +/- 3.0 │        128.5 +/- 120.9 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

🆙 Scale up!

  • Happy with your preview data?

  • Use the create method to submit larger Data Designer generation jobs.

Python
1results = data_designer.create(config_builder, num_records=10, dataset_name="tutorial-1")
2
Output
[02:04:06] [INFO] OpenTelemetry metrics available at http://127.0.0.1:9464/metrics
[02:04:06] [INFO] 🎨 Creating Data Designer dataset
[02:04:06] [INFO]   |-- 🔒 Jinja rendering engine: secure
[02:04:06] [INFO] ✅ Validation passed
[02:04:06] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[02:04:06] [INFO] Skipping model health checks because DATA_DESIGNER_SKIP_MODEL_HEALTH_CHECKS=1
[02:04:06] [INFO] ⚡ Using async task-queue builder
[02:04:06] [INFO] 📝 llm-text model config for column 'product_name'
[02:04:06] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[02:04:06] [INFO]   |-- model alias: 'nemotron-lightning'
[02:04:06] [INFO]   |-- model provider: 'nvidia'
[02:04:06] [INFO]   |-- inference parameters:
[02:04:06] [INFO]   |  |-- generation_type=chat-completion
[02:04:06] [INFO]   |  |-- max_parallel_requests=4
[02:04:06] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[02:04:06] [INFO]   |  |-- temperature=1.00
[02:04:06] [INFO]   |  |-- top_p=0.95
[02:04:06] [INFO]   |  |-- max_tokens=2048
[02:04:06] [INFO] 📝 llm-text model config for column 'customer_review'
[02:04:06] [INFO]   |-- model: 'nvidia/nemotron-3.5-lightning-30b-a3b'
[02:04:06] [INFO]   |-- model alias: 'nemotron-lightning'
[02:04:06] [INFO]   |-- model provider: 'nvidia'
[02:04:06] [INFO]   |-- inference parameters:
[02:04:06] [INFO]   |  |-- generation_type=chat-completion
[02:04:06] [INFO]   |  |-- max_parallel_requests=4
[02:04:06] [INFO]   |  |-- extra_body={'chat_template_kwargs': {'enable_thinking': False}}
[02:04:06] [INFO]   |  |-- temperature=1.00
[02:04:06] [INFO]   |  |-- top_p=0.95
[02:04:06] [INFO]   |  |-- max_tokens=2048
[02:04:06] [INFO] ⚡️ Async generation: 2 column(s) (column 'product_name', column 'customer_review'), 20 tasks across 1 row group(s)
[02:04:06] [INFO] 🚀 (1/1) Dispatching with 10 records
[02:04:06] [INFO] 🎲 (1/1) Preparing samplers to generate 10 records across 6 columns
[02:04:07] [WARNING] Observed retryable model-task error: kind=rate_limit, http_status=429; the row task will be deferred.
[02:04:15] [INFO] 📊 Progress [8.3s]:
[02:04:15] [INFO]   |-- 🥱 column 'product_name': 4/10 (40%) 0.5 rec/s
[02:04:15] [INFO]   |-- 🐱 column 'customer_review': 1/10 (10%) 0.1 rec/s
[02:04:22] [INFO] 📊 Progress [15.5s]:
[02:04:22] [INFO]   |-- 😐 column 'product_name': 6/10 (60%) 0.4 rec/s
[02:04:22] [INFO]   |-- 🐱 column 'customer_review': 1/10 (10%) 0.1 rec/s
[02:04:29] [INFO] 📊 Progress [22.6s]:
[02:04:29] [INFO]   |-- 😐 column 'product_name': 7/10 (70%) 0.3 rec/s
[02:04:29] [INFO]   |-- 😺 column 'customer_review': 3/10 (30%) 0.1 rec/s
[02:04:30] [INFO] 🔄 (1/1) Salvaging 6 deferred task(s)
[02:04:37] [INFO] 📊 Progress [30.9s]:
[02:04:37] [INFO]   |-- 😊 column 'product_name': 9/10 (90%) 0.3 rec/s
[02:04:37] [INFO]   |-- 😸 column 'customer_review': 6/10 (60%) 0.2 rec/s
[02:04:49] [INFO] 🔄 (1/1) Preserving 3 retryable task(s); records will keep retrying.
[02:04:57] [WARNING] Provider showing degraded performance: 50% of last 20 task outcomes were retryable errors (rate_limit=10, success=10); cumulative outcomes: rate_limit=16, success=17; deferred_tasks=1. Run may take longer than expected; salvage will retry these.
[02:05:23] [INFO] 📊 Progress [76.9s]:
[02:05:23] [INFO]   |-- 🤩 column 'product_name': 10/10 (100%) 0.1 rec/s
[02:05:23] [INFO]   |-- 😼 column 'customer_review': 8/10 (80%) 0.1 rec/s
[02:05:31] [INFO] 📊 Progress [84.9s]:
[02:05:31] [INFO]   |-- 🤩 column 'product_name': 10/10 (100%) 0.1 rec/s
[02:05:31] [INFO]   |-- 🦁 column 'customer_review': 10/10 (100%) 0.1 rec/s
[02:05:31] [INFO] ✅ Async generation complete [84.9s]: 20 ok, 0 failed across 2 column(s)
[02:05:32] [INFO] 📊 Model usage summary:
[02:05:32] [INFO]   |-- model: nvidia/nemotron-3.5-lightning-30b-a3b
[02:05:32] [INFO]   |-- tokens: input=1758, output=2649, reasoning=0, total=4407, tps=51
[02:05:32] [INFO]   |-- requests: success=20, failed=27, total=47, rpm=33
[02:05:32] [INFO] 📐 Measuring dataset column statistics:
[02:05:32] [INFO]   |-- 🎲 column: 'product_category'
[02:05:32] [INFO]   |-- 🎲 column: 'product_subcategory'
[02:05:32] [INFO]   |-- 🎲 column: 'target_age_range'
[02:05:32] [INFO]   |-- 🎲 column: 'customer'
[02:05:32] [INFO]   |-- 🎲 column: 'number_of_stars'
[02:05:32] [INFO]   |-- 🎲 column: 'review_style'
[02:05:32] [INFO]   |-- 📝 column: 'product_name'
[02:05:32] [INFO]   |-- 📝 column: 'customer_review'
Python
1# Load the generated dataset as a pandas DataFrame.
2dataset = results.load_dataset()
3 
4dataset.head()
5
Output
product_category product_subcategory target_age_range customer number_of_stars review_style product_name customer_review
0 Books Fiction 25-35 {'uuid': '935e2c32-88b7-4d6e-a094-df033fcf9ca6... 5 brief Ink & Idle I'm obsessed with Ink & Idle! It's the perfect...
1 Electronics Accessories 25-35 {'uuid': 'cbafc5a2-e531-4b7b-b5a7-93c706cc5e76... 2 brief Quantum Pulse Band The product arrived well-packaged, but feels c...
2 Electronics Cameras 25-35 {'uuid': '64bbca3a-a9f9-4465-9a7d-aef82b7be853... 5 rambling SnapStream Pocket Oh my goodness you guys I am literally shaking...
3 Books Classics 35-50 {'uuid': '4274ad42-7397-43a4-9180-a41b23c7169f... 3 detailed Lexicon Legends I recently purchased Lexicon Legends from the ...
4 Books Non-Fiction 65+ {'uuid': '14e6af08-c8c8-4209-a188-6ff6a20f8e67... 2 brief Golden Chapter Guide Found it promising at first, but ultimately un...
Python
1# Load the analysis results into memory.
2analysis = results.load_analysis()
3 
4analysis.to_report()
5
Output
──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────

                                                                                                                   
                                                 Dataset Overview                                                  
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 10                              │ 8                               │ 100.0%                                      │
└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                🎲 Sampler Columns                                                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column name                    ┃       data type ┃            number unique values ┃               sampler type ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_category               │          string │                       3 (30.0%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ product_subcategory            │          string │                       7 (70.0%) │                subcategory │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ target_age_range               │          string │                       5 (50.0%) │                   category │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ customer                       │            dict │                     10 (100.0%) │          person_from_faker │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ number_of_stars                │             int │                       5 (50.0%) │                    uniform │
├────────────────────────────────┼─────────────────┼─────────────────────────────────┼────────────────────────────┤
│ review_style                   │          string │                       4 (40.0%) │                   category │
└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘
                                                                                                                   
                                                                                                                   
                                                📝 LLM-Text Columns                                                
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃
┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃
┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ product_name          │        string │                10 (100.0%) │      73.5 +/- 0.5 │            4.0 +/- 0.8 │
├───────────────────────┼───────────────┼────────────────────────────┼───────────────────┼────────────────────────┤
│ customer_review       │        string │                10 (100.0%) │      68.0 +/- 1.5 │        200.5 +/- 255.0 │
└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘
                                                                                                                   
                                                                                                                   
╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮
│                                                                                                                 │
│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │
│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │
│                                                                                                                 │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                                                                                                                   
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────

⏭️ Next Steps

Now that you've seen the basics of Data Designer, check out the following notebooks to learn more about: