The Basics

View as Markdown

🎨 Data Designer Tutorial: The Basics

📚 What you'll learn

This notebook demonstrates the basics of Data Designer by generating a simple product review dataset.

📦 Import Data Designer

  • data_designer.config provides access to the configuration API.

  • DataDesigner is the main interface for data generation.

Python
1import data_designer.config as dd
2from data_designer.interface import DataDesigner
3

⚙️ Initialize the Data Designer interface

  • DataDesigner is the main object responsible for managing the data generation process.

  • When initialized without arguments, the default model providers are used.

Python
1data_designer = DataDesigner()
2

🎛️ Define model configurations

  • Each ModelConfig defines a model that can be used during the generation process.

  • The "model alias" is used to reference the model in the Data Designer config (as we will see below).

  • The "model provider" is the external service that hosts the model (see the model config docs for more details).

  • By default, we use build.nvidia.com as the model provider.

Python
1# This name is set in the model provider configuration.
2MODEL_PROVIDER = "nvidia"
3
4# The model ID is from build.nvidia.com.
5MODEL_ID = "nvidia/nemotron-3-nano-30b-a3b"
6
7# We choose this alias to be descriptive for our use case.
8MODEL_ALIAS = "nemotron-nano-v3"
9
10model_configs = [
11 dd.ModelConfig(
12 alias=MODEL_ALIAS,
13 model=MODEL_ID,
14 provider=MODEL_PROVIDER,
15 inference_parameters=dd.ChatCompletionInferenceParams(
16 temperature=1.0,
17 top_p=1.0,
18 max_tokens=2048,
19 extra_body={"chat_template_kwargs": {"enable_thinking": False}},
20 ),
21 )
22]
23

🏗️ Initialize the Data Designer Config Builder

  • The Data Designer config defines the dataset schema and generation process.

  • The config builder provides an intuitive interface for building this configuration.

  • The list of model configs is provided to the builder at initialization.

Python
1config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
2

🎲 Getting started with sampler columns

  • Sampler columns offer non-LLM based generation of synthetic data.

  • They are particularly useful for steering the diversity of the generated data, as we demonstrate below.


You can view available samplers using the config builder's info property:

Python
1config_builder.info.display("samplers")
2

Let's start designing our product review dataset by adding product category and subcategory columns.

Python
1config_builder.add_column(
2 dd.SamplerColumnConfig(
3 name="product_category",
4 sampler_type=dd.SamplerType.CATEGORY,
5 params=dd.CategorySamplerParams(
6 values=[
7 "Electronics",
8 "Clothing",
9 "Home & Kitchen",
10 "Books",
11 "Home Office",
12 ],
13 ),
14 )
15)
16
17config_builder.add_column(
18 dd.SamplerColumnConfig(
19 name="product_subcategory",
20 sampler_type=dd.SamplerType.SUBCATEGORY,
21 params=dd.SubcategorySamplerParams(
22 category="product_category",
23 values={
24 "Electronics": [
25 "Smartphones",
26 "Laptops",
27 "Headphones",
28 "Cameras",
29 "Accessories",
30 ],
31 "Clothing": [
32 "Men's Clothing",
33 "Women's Clothing",
34 "Winter Coats",
35 "Activewear",
36 "Accessories",
37 ],
38 "Home & Kitchen": [
39 "Appliances",
40 "Cookware",
41 "Furniture",
42 "Decor",
43 "Organization",
44 ],
45 "Books": [
46 "Fiction",
47 "Non-Fiction",
48 "Self-Help",
49 "Textbooks",
50 "Classics",
51 ],
52 "Home Office": [
53 "Desks",
54 "Chairs",
55 "Storage",
56 "Office Supplies",
57 "Lighting",
58 ],
59 },
60 ),
61 )
62)
63
64config_builder.add_column(
65 dd.SamplerColumnConfig(
66 name="target_age_range",
67 sampler_type=dd.SamplerType.CATEGORY,
68 params=dd.CategorySamplerParams(values=["18-25", "25-35", "35-50", "50-65", "65+"]),
69 )
70)
71
72# Optionally validate that the columns are configured correctly.
73data_designer.validate(config_builder)
74

Next, let's add samplers to generate data related to the customer and their review.

Python
1config_builder.add_column(
2 dd.SamplerColumnConfig(
3 name="customer",
4 sampler_type=dd.SamplerType.PERSON_FROM_FAKER,
5 params=dd.PersonFromFakerSamplerParams(age_range=[18, 70], locale="en_US"),
6 )
7)
8
9config_builder.add_column(
10 dd.SamplerColumnConfig(
11 name="number_of_stars",
12 sampler_type=dd.SamplerType.UNIFORM,
13 params=dd.UniformSamplerParams(low=1, high=5),
14 convert_to="int", # Convert the sampled float to an integer.
15 )
16)
17
18config_builder.add_column(
19 dd.SamplerColumnConfig(
20 name="review_style",
21 sampler_type=dd.SamplerType.CATEGORY,
22 params=dd.CategorySamplerParams(
23 values=["rambling", "brief", "detailed", "structured with bullet points"],
24 weights=[1, 2, 2, 1],
25 ),
26 )
27)
28
29data_designer.validate(config_builder)
30

🦜 LLM-generated columns

  • The real power of Data Designer comes from leveraging LLMs to generate text, code, and structured data.

  • When prompting the LLM, we can use Jinja templating to reference other columns in the dataset.

  • As we see below, nested json fields can be accessed using dot notation.

Python
1config_builder.add_column(
2 dd.LLMTextColumnConfig(
3 name="product_name",
4 prompt=(
5 "You are a helpful assistant that generates product names. DO NOT add quotes around the product name.\n\n"
6 "Come up with a creative product name for a product in the '{{ product_category }}' category, focusing "
7 "on products related to '{{ product_subcategory }}'. The target age range of the ideal customer is "
8 "{{ target_age_range }} years old. Respond with only the product name, no other text."
9 ),
10 model_alias=MODEL_ALIAS,
11 )
12)
13
14config_builder.add_column(
15 dd.LLMTextColumnConfig(
16 name="customer_review",
17 prompt=(
18 "You are a customer named {{ customer.first_name }} from {{ customer.city }}, {{ customer.state }}. "
19 "You are {{ customer.age }} years old and recently purchased a product called {{ product_name }}. "
20 "Write a review of this product, which you gave a rating of {{ number_of_stars }} stars. "
21 "The style of the review should be '{{ review_style }}'. "
22 "Respond with only the review, no other text."
23 ),
24 model_alias=MODEL_ALIAS,
25 )
26)
27
28data_designer.validate(config_builder)
29

🔁 Iteration is key – preview the dataset!

  1. Use the preview method to generate a sample of records quickly.

  2. Inspect the results for quality and format issues.

  3. Adjust column configurations, prompts, or parameters as needed.

  4. Re-run the preview until satisfied.

Python
1preview = data_designer.preview(config_builder, num_records=2)
2
Python
1# Run this cell multiple times to cycle through the 2 preview records.
2preview.display_sample_record()
3
Python
1# The preview dataset is available as a pandas DataFrame.
2preview.dataset
3

📊 Analyze the generated data

  • Data Designer automatically generates a basic statistical analysis of the generated data.

  • This analysis is available via the analysis property of generation result objects.

Python
1# Print the analysis as a table.
2preview.analysis.to_report()
3

🆙 Scale up!

  • Happy with your preview data?

  • Use the create method to submit larger Data Designer generation jobs.

Python
1results = data_designer.create(config_builder, num_records=10, dataset_name="tutorial-1")
2
Python
1# Load the generated dataset as a pandas DataFrame.
2dataset = results.load_dataset()
3
4dataset.head()
5
Python
1# Load the analysis results into memory.
2analysis = results.load_analysis()
3
4analysis.to_report()
5

⏭️ Next Steps

Now that you've seen the basics of Data Designer, check out the following notebooks to learn more about: