Data Designer in NeMo Studio

View as Markdown

Data Designer lets you generate and transform datasets visually — no JSON to write. You start from a template, an AI-drafted config, a clone of an earlier job, or an empty canvas; lay out columns in the builder and wire them together by referencing one another; preview a handful of rows; then run the full job and inspect, profile, edit, and split the result.

Studio is still in early development. Many features are missing or should be expected to change.

This page walks through the full workflow:

  1. View existing jobs
  2. Start a new fileset
  3. Build a recipe with the builder
  4. Add columns
  5. Configure column properties
  6. Add and configure models
  7. Preview a job
  8. Create the job
  9. View job details after completion
  10. Edit rows after completion
  11. Transform the dataset into another schema
  12. Split the dataset

View jobs

Navigate to Data > Datasets > Data Designer in the workspace sidebar to open the jobs list. The list shows every Data Designer job in the current workspace with its status, and supports searching by name, filtering by status, and narrowing by creation or update date range.

Selecting a job opens its details page. Each row also has a quick-actions menu with View details, Clone, Cancel (when the job is still cancellable), and Delete.

Job status values match the platform-wide Jobs statuses (Created, Pending, Active, Completed, Error, Cancelled, and so on).


Start a new fileset

Click New Job on the jobs list to open the Create a fileset start screen. Pick a start option; a detail area for that option opens below the tiles, and Continue at the bottom of the page takes you into the builder.

Start optionWhat it does
Describe with AIDescribe the dataset in plain language and have a model draft the columns, prompts, and model configs for you.
Start from a templatePick a prebuilt recipe card; the builder opens pre-seeded with that template’s columns and models.
Build from scratchOpens the builder with an empty schema list.

Continue stays disabled until the chosen option has everything it needs — a selected recipe card for a template, a validated draft for AI — and the reason is shown next to the button.

A fourth path skips this screen entirely: Clone on any existing job (from the jobs list or the job details page) opens the builder pre-filled with that job’s row count, columns, and models. See Clone an existing job.

Describe with AI

The Describe with AI panel has two halves: the prompt on the left, and the verdict on the draft on the right.

  • Model — the model that drafts the config. It must support tool calling, since the config comes back as a tool call. The same model is also written into every model config in the draft, so the LLM columns generate with a model that is known to exist in your workspace.
  • What do you want to generate? — a plain-language description of the dataset (row count, columns, labels, sampling, and so on). While the field is empty it offers a few example prompts (phishing email triage, support ticket routing, refund policy Q&A) as pills; picking one fills the field with a complete, generation-ready description you can edit.
  • Generate — sends the request. Once you have a draft, the button becomes Regenerate.

Every draft is checked against the builder’s own rules before it can be loaded, so a broken config never reaches the canvas. The result panel shows one of:

  • Valid job config — plus a summary of the fileset name, record count, column and model counts, and each generated column with its type. Continue is now enabled.
  • Errors — the draft can’t be loaded (for example, no columns the builder can edit, or an LLM column pointing at a model_alias no model defines). Fix these errors sends the draft and the error list back to the same model for a repair pass.
  • Warnings — the draft is loadable but something was adjusted: a model was substituted, a column type the builder can’t edit was skipped, or a non-positive record count was defaulted to 100. Fix these warnings runs the same repair pass.

View config opens a side panel with the model’s raw tool-call output, so you can inspect a draft before loading it — or work out why one was rejected. It shows exactly what the model returned, so it can differ from the config that lands on the canvas when a model had to be substituted.

Once loaded, the generated config is fully editable in the builder like any other recipe — it is not opaque JSON.

Templates

Selecting Start from a template shows the recipe cards. Each card carries a use-case tag that follows you into the builder toolbar.

RecipeTagWhat it generates
Phishing email triage (evaluation set)EvaluationLabeled synthetic emails for the email-phishing-analyzer benchmark; the label is sampled, not model-authored.
Phishing analyzer fine-tuning (SFT)Fine-tuningPrompt–completion pairs teaching a small model the phishing-analyzer task, with a validated JSON verdict as output.
Instruction fine-tuning (SFT)Fine-tuningInstruction–response pairs: a sampled topic, an LLM-generated instruction, and a model answer.
All samplers (showcase)ShowcaseOne column per previewable sampler sub-type, seeded with valid params.
Code generation + validation (Python)Fine-tuningPython coding challenges with LLM-generated solutions and a pass/fail validation column.
Structured data extractionFine-tuningFree-form text paired with its structured JSON representation.
Preference pairs (reward modeling)AlignmentAn instruction with chosen and rejected answers plus an LLM judge score, for DPO and reward modeling.
Semantic search datasetRetrievalQuery–passage pairs with vector embeddings. Requires an embedding model under the embedder alias.
Expression transforms (no LLM)TransformDerived columns computed via Jinja2 expressions. No LLM calls, so it previews instantly.

Clone an existing job

Choose Clone from any job’s quick-actions menu. The builder opens with the source job’s columns, models, and row count already loaded, under the name <job name>-copy. Edit anything you like and create it as a new job — the source job is untouched.


The builder

The builder is a three-pane layout with a toolbar across the top:

  • Toolbar (top) — fileset name, column count, template badge, view toggle, row count, and the Preview and Create fileset actions.
  • Left palette — a segmented Columns / Models switch for adding columns or models to the recipe.
  • Center pane — your columns, shown in whichever view mode is active (see View modes below).
  • Right config pane — properties for whichever column or model is currently selected. With nothing selected it prompts you to pick a column or model, or add one from the left.

View modes

A segmented control in the toolbar lets you switch between two views of your columns:

ViewIconDescription
List (default)List-tree iconA flat, top-to-bottom list of columns, ordered by generation order rather than the order you added them. Each row shows an icon badge, the column name, a type badge, a one-line summary, and inline {{ reference }} tags for its dependencies. Click a row to select it; use the trash icon at the right to delete it.
CanvasSpline iconA directed acyclic graph (DAG) where columns appear as nodes and dependencies as arrows. Use this view to see the full dependency graph at a glance.

Both views open the same config pane when a column is selected. All edits made in one view are immediately reflected in the other.

The toolbar

  • Name — the fileset name. Click the pencil icon to rename it (defaults to untitled-dataset, the template id when starting from a template, or <job name>-copy when cloning). A name is required to create the job.
  • Column count — a live count of columns in the recipe.
  • View toggle — switches between the list and canvas views (see View modes).
  • Rows — the number of records the full run generates. Must be a whole number of at least 1.
  • Preview N rows — runs a small sample. N is your row count, capped at 10. See Preview a job. While a preview runs, a Stop button appears next to it to abort the run.
  • Create fileset — validates the recipe and submits the job. See Create the job.

How columns connect

Column dependencies are tracked automatically. Whenever one column refers to another — for example, a prompt that mentions {{ other_column }} — the builder records that relationship. In the list view these show as inline {{ name }} tags on the dependent row; in the canvas view they appear as arrows between nodes.


Add columns

Open the Columns tab in the left palette and pick a column type. A search box filters the catalog across column names and descriptions. The catalog is grouped to mirror the Data Designer column concepts:

GroupColumn types
SamplerUUID, Category, Subcategory, Uniform, Gaussian, Bernoulli, Bernoulli Mixture, Binomial, Poisson, Scipy, Person, Datetime, Timedelta — generate values from statistical distributions or built-in generators, no LLM required.
GenerateLLM-Text, LLM-Code, LLM-Structured (JSON to a schema), LLM-Judge (score content), Image, Embedding — produced by a model.
TransformExpression — a Jinja2 transform evaluated per row, no LLM.
ValidateValidation — check target columns against rules (code, local callable, or remote validator).
Data & customSeed Dataset (bootstrap rows from a file), Custom (Python function logic).

Adding a column appends it to the list (and centers the new node in canvas view), selects it, and opens its config pane immediately. A default, unique column name (for example category_1) is assigned; you can rename it in the config pane.

Only one Seed Dataset column is supported per recipe. After you add one, the Seed Dataset card is disabled in the palette and explains why on hover.

To remove a column, use the trash icon on its row in list view, or select its node and use Remove in the config pane (both views).


Configure column properties

Select any column to open its properties in the right config pane. Every column type has its own set of fields; the builder only shows the fields relevant to the selected type.

Common patterns:

  • Name — must be unique and use letters, numbers, and underscores, and must not start with a number.
  • Prompt (Generate columns) — a Jinja2 template. Reference other columns with {{ column_name }}; those references create the DAG edges.
  • Model alias (Generate columns) — the alias of a configured model to generate with.
  • Sampler params — each sampler sub-type exposes its own parameters (for example Category takes comma-separated values and optional weights; Gaussian takes mean and standard deviation; Datetime takes a start/end range).
  • Field checks — the builder validates your entries as you go and flags anything that isn’t valid, such as a number field left with text or a malformed entry.

Empty required fields, invalid values, and duplicate or invalid names are all called out when you preview or create the job.


Add and configure models

Generate columns (LLM-Text, LLM-Code, LLM-Structured, LLM-Judge, Image, Embedding) reference a model by alias. Switch the left palette to the Models tab to add and manage the models your recipe uses.

  • Add a model — pick a model from the workspace model groups and a provider; it’s added with an editable alias.
  • Configure a model — select it to open the model config pane, which has:
    • Alias — how LLM columns refer to this model. Must be unique. Changing the model re-derives the alias from the new model name and updates every column that referenced the old alias.
    • Model — the workspace model, along with its inference parameters.
    • Max parallel requests — how many generation requests this model may have in flight at once. Lower it if your inference provider rate-limits the job.
    • Remove model — deletes it from the recipe.
  • Wire it up — set a Generate column’s Model alias field to the model’s alias.

When you open a template, its seeded models are auto-filled from the workspace model list once that list finishes loading, so you usually don’t need to reselect them. Configs drafted by Describe with AI are pointed at the model you picked in that panel.


Preview a job

Click Preview N rows in the toolbar to generate a small sample with the current recipe (N is your row count, capped at 10). This is a fast way to sanity-check prompts, sampler settings, and dependencies before committing to a full run. Use Stop to abort a preview that’s still running.

The details panel below the toolbar opens automatically and shows:

  • Validation issues — anything blocking the run (missing required fields, malformed JSON/numbers, duplicate or invalid names, no columns added, invalid row count). Preview and Create are both blocked until these are resolved.
  • Preview logs — the sampled output, streamed as the preview runs.

You can collapse the panel to focus on the builder; runs that produce output re-open it.


Create the job

Click Create fileset in the toolbar. The builder validates the whole recipe first (columns, models, name, and row count). If anything fails, the errors appear in the details panel and the job is not submitted.

On success, the job is created with your configured name, row count, and generated Data Designer config, and Studio navigates to the new job’s details page. If job creation returns an error, it’s shown in the details panel.


View job details after completion

The job details page shows the job name, status badge, description, and created/updated timestamps, plus a job actions menu with Transform (see Transform the dataset), Split (see Split the dataset), View config (opens the generated Data Designer config), Clone, Cancel, and Delete. Transform and Split are disabled until the job has produced .json, .jsonl, or .parquet output.

Details are organized into four tabs. The page opens on Profile for a job in a terminal state and on Logs for one that’s still running.

TabWhat it shows
ProfileA dataset profile once the job reaches a terminal state — completeness (rows generated vs. target) and per-column statistics cards. Before then it points you at the Logs tab.
DataThe generated data files, rendered as editable rows. See Edit rows after completion.
Output filesThe raw files in the job’s output fileset, with preview and download.
LogsLive job logs, streamed while the job runs and available afterwards.

Edit rows after completion

The Data tab shows the job’s generated data in an in-browser row editor. It lists the output data files (.parquet, .jsonl, .json, .csv); when there’s more than one, a file selector lets you switch between them.

In the editor you can:

  • Edit a cell or row — open a row to edit its fields; changes are applied to the in-memory table.
  • Add Row — append a new empty row.
  • Duplicate or Delete a row.
  • Search / filter rows.
  • Download the current rows.
  • Save File — persist your edits back to the fileset. The action is enabled only when there are unsaved changes.

How your edits are saved depends on the file format:

  • Text formats (.json, .jsonl, .csv) are updated in place.
  • Parquet files can’t be rewritten in the browser, so your edits are saved to a new .edited.jsonl file alongside the original, leaving the source .parquet untouched. The tab then defaults to that edited file.

Files larger than 8 MB (non-Parquet) can’t be edited in the browser. Download them to edit locally.


Transform the dataset into another schema

A finished dataset rarely lands in the exact shape the next tool wants. Transform rewrites each row into another schema — renaming, nesting, dropping, and combining fields — without regenerating any data.

Choose Transform from the job actions menu on the details page. It is available once the job has produced a .json, .jsonl, or .parquet output file.

Under the hood, the transform launches a second Data Designer job that only maps fields: the file you pick becomes the seed, the job declares no generated columns, and a schema_transform processor rewrites every row. Nothing is generated and no model is called, so the transform costs no inference. The result is written to processors-files/<output name>/ in the new job’s fileset, and Studio navigates to the new job when it is created.

Pick a target format

The Target format cards decide which fields the mapping asks you for:

FormatWhat it produces
Evaluation TasksTasks the Evaluator can run an agent against: id, intent, inputs.instruction, and an optional grader-only reference.expected.
MessagesA two-turn messages array — a user turn and an assistant turn — the usual shape for supervised fine-tuning. The role values are filled in for you.
CustomThe raw output schema, one key at a time. Starts as a passthrough of the source columns, so renaming or dropping a few fields is an edit rather than a rewrite from scratch.

A format is a Studio-side convenience: it names the fields the target consumer expects and pre-fills the underlying template. Anything a format can’t express is still reachable through Custom.

Map the fields

Studio reads the source file’s columns and guesses a mapping from the column names, so a well-named dataset is often already mapped when the modal opens.

Each field row shows whether it is Required or Optional, a source column picker, and a { } toggle that swaps the picker for a raw Jinja2 input — use it for filters, concatenation, or literal values the picker can’t express. Leaving a field blank drops it from the output entirely instead of writing an empty string. Required fields with no source block the transform and are called out in a banner.

Custom replaces the field rows with a key/template grid. Keys accept dot paths and numeric segments, so reference.expected nests an object and messages.0.content builds an array. Filling in the last row grows the grid; the available source columns are listed below it as {{ column }} chips.

Template values are Jinja2:

  • {{ column }} inserts that column’s value for the row. Text outside the braces is kept, so Ticket {{ id }}: {{ summary }} is one field.
  • Text with no braces is a constant — every row gets the same value.
  • Filters transform a value: {{ topic | upper }}, {{ score | int }}.
  • {{ notes | default('none') }} covers an undefined column; add true as a second argument to replace empty values too.

If the format needs a unique identifier per row and no source column matches, the picker offers a generated option instead. The job adds that column as a UUID sampler, one value per row — still no inference. A constant would be identical on every row, which silently collapses the output.

Preview, then create

The before / after panel renders a single source row through the current mapping so you can check it against real data before anything is written. Step through rows with the row selector. Complex Jinja2 — filters, blocks, helpers — is only approximated in the browser; the transform applies it exactly when it runs.

The bottom of the modal sets:

  • Job name — the new transform job’s name, seeded from the source job and the format.
  • Output name — the processor name, which is also the directory the output is written to.
  • Rows — how many rows to read from the source, defaulting to the source job’s record count. Asking for more rows than the source has restarts the reader at the top of the file and duplicates rows; Studio warns when you do.

Closing the modal with an unsubmitted mapping asks you to confirm before discarding it.

Transform a single fileset file in place

The same mapping is available outside Data Designer. On any read/write fileset, choose Transform from a file’s quick-actions menu to rewrite that file through a format and field mapping.

That path applies the mapping in the browser and overwrites the file, so no job is created — and because the rows are re-serialized as JSONL, only .jsonl files can be transformed in place. Use the Data Designer transform above for .json, .csv, and .parquet sources, or for keeping the original alongside the result.


Split the dataset

Once a job has produced .json, .jsonl, or .parquet output, the Split button on the details page opens the Create Split modal, which carves the generated data into the training, validation, and test subsets you need to fine-tune and evaluate a model.

  • Source file — which output file to split.
  • Split percentage — 80/10/10 (recommended), 60/20/20, 80/20, 90/10, or a custom split.
  • Distribution typeRandom when row order doesn’t affect model performance, or Sequential for tasks with sequence dependencies. Advanced options let you set a seed or a sort key.

The splits are written back into the job’s output fileset.