Data Designer

Data Designer on NeMo Platform enables high-quality synthetic data generation through the NeMo Data Designer plugin. You can submit workloads to a running NeMo Services cluster using the CLI or SDK.

Overview

Data Designer is a framework for orchestrating complex synthetic data generation workflows. It coordinates LLM calls, manages dependencies between data fields, handles batching and parallelization, and validates generated data against specifications.

The plugin is built on the open-source NVIDIA NeMo Data Designer library (GitHub). The library provides the configuration and generation engine; the plugin provides CLI, SDK, Data Designer API, Jobs, Files API, Secrets API, and Inference Gateway API integration.

How It Works

Data Designer separates configuration from execution.

1. Build Configurations

Use data_designer.config to define the dataset you want to generate:

1
import data_designer.config as dd
2
3
# Define models
4
model_configs = [
5
    dd.ModelConfig(
6
        provider="default/nvidia-build",
7
        model="nvidia/nemotron-3-nano-30b-a3b",
8
        alias="text",
9
    )
10
]
11
12
# Build configuration
13
config_builder = dd.DataDesignerConfigBuilder(model_configs)
14
config_builder.add_column(dd.SamplerColumnConfig(...))
15
config_builder.add_column(dd.LLMTextColumnConfig(...))

Configuration code describes the dataset schema, columns, dependencies, constraints, seed data, processors, profilers, and inference settings.

Learn more: See the library documentation for comprehensive guides on column types, samplers, constraints, and advanced features.

2. Execute on the Platform

The dataset defined by your configuration is executed on NeMo Platform, integrating with these NeMo Services APIs:

Next Steps