> This page is for version 0.10. > For other versions, use one of these documentation indexes: > - Latest (default): https://docs.nvidia.com/nemo-helix/latest/llms.txt > - 0.10: https://docs.nvidia.com/nemo-helix/v0.10/llms.txt > - 0.9: https://docs.nvidia.com/nemo-helix/v0.9/llms.txt > - 0.8: https://docs.nvidia.com/nemo-helix/v0.8/llms.txt > - 0.7: https://docs.nvidia.com/nemo-helix/v0.7/llms.txt > - 0.6: https://docs.nvidia.com/nemo-helix/v0.6/llms.txt > - 0.5: https://docs.nvidia.com/nemo-helix/v0.5/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-helix/_mcp/server. # Dataset Format Requirements Use the following guidelines to prepare your training dataset for the supported models. ## Dataset Preparation Guidelines * **File Format**: Save your training data as `.jsonl` files (one JSON object per line). * **Validation**: Each record is automatically validated against the appropriate schema when training begins. The required format depends on `training.training_type` (for example, `sft`) specified in your job configuration. For dataset creation tutorials, refer to [Format Training Dataset](/nemo-helix/v0.10/documentation/customizer-reference/tutorials/format-training-dataset). ## Prepare Data Designer Output or a Table `nemo customization dataset prepare` turns a Parquet, CSV or JSONL table, such as Data Designer output, into a training dataset in the format you name with `--format`. It maps the columns, splits the rows with a seeded shuffle, and uploads the splits to a new `dataset` fileset, printing the references to put in your job: ```bash # A local file or directory, or a fileset nemo customization dataset prepare ./rows.parquet --name sft-data --format chat nemo customization dataset prepare default/my-table --name dpo-data --format preference ``` To prepare the output of a Data Designer job, download its `artifacts` result and pass the `parquet-files` directory inside it: ```bash nemo jobs results download artifacts --job my-dd-job --workspace default -o artifacts.tar tar -xf artifacts.tar nemo customization dataset prepare "$(find . -type d -name parquet-files)" --name sft-data --format chat ``` | `--format` | Each row | Trains on | Columns read by default | | ------------------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------- | | `chat` | [OpenAI Chat](#openai-chat-model) `messages` | Automodel `sft` and `distillation`; Unsloth with `dataset.apply_chat_template: true` | `instruction` or `prompt`, and `response` or `completion`; or a `messages` column | | `preference` | [Preference pair](#preference-pairs) | NeMo-RL `dpo` | `instruction` or `prompt`, `chosen`, `rejected` | | `prompt-completion` | [Prompt and completion](#basic-prompt-completion), trained as raw text with no chat template | Automodel; Unsloth | as `chat` | | `embedding` | [Embedding triplet](#embedding-model) | Automodel with `training.recipe` `bi_encoder` or `cross_encoder` | `query`, `pos_doc`, `neg_doc` (a list) | | `text` | [Text](#text) | Unsloth with `dataset.text_field: text` | `text` | The command does not prepare GRPO datasets: GRPO trains on NeMo Gym rows that each environment builds. The Studio "Instruction fine-tuning" and "Preference pairs" templates need no column flags. For other column names, pass the flag for the field: `--prompt-column`, `--response-column`, `--system-column` (adds a system message, for `chat` and `preference`; with it, every `preference` prompt is a message list, so the column has one type), `--messages-column`, `--chosen-column`, `--rejected-column`, `--query-column`, `--positive-column`, `--negatives-column` or `--text-column`. A flag the format does not use, or a column the table does not have, fails before anything is uploaded. `--input-format` reads files whose extension does not give their format, or one format from a directory that mixes them. Every format writes `train.jsonl` and `validation.jsonl`. With `--val-fraction 0` there is no `validation.jsonl`, and training holds out its own validation rows. The test split (`--test-fraction`, 0 by default for `preference` and 0.1 otherwise) goes to a separate `-test` fileset, so training never reads it. `--val-fraction` (default 0.1) and `--seed` (default 42) set the rest; the same seed always gives the same split, and a fraction above 0 gets at least one row even from a small table. Rows with an empty required field, such as a failed generation, are dropped and counted; an empty system prompt is left out and empty negative documents are skipped. ## Dataset Formats The following sections describe the required schema and example dataset entries for each model type. ### Embedding Model For embedding model training, the following additional guidelines apply: * **Negative Examples**: Provide multiple negative documents per query for better contrastive learning. * **Content Quality**: Ensure positive documents are semantically relevant to the query. * **JSONL Format**: The supported embedding models require training data in JSONL format with a specific triplet structure for contrastive learning. #### Required Schema Each line in your JSONL file must contain a JSON object with these required fields: * **`query`** (string): The query text to use as an anchor for similarity learning. * **`pos_doc`** (string): A document that should match positively with the query. * **`neg_doc`** (array of strings): One or more documents that should not match with the query. Retrieval Stage 1 run without mining (`enable_mining: false`) emits an **empty** list. Automodel `bi_encoder` / `cross_encoder` jobs then fail at collate time with `neg_doc must contain at least 1 document to sample N negatives` (N defaults to 4\). Mine hard negatives before submitting, or provide your own non-empty lists. Do not lower `training.retrieval.train_n_passages` to hide an unmined fileset. #### Example Dataset Entry ```json { "query": "3D trajectory recovery for tracking multiple objects", "pos_doc": "Recursive Estimation of Motion, Structure, and Focal Length", "neg_doc": [ "Characterization of 1.2 kV SiC super-junction SBD implemented by trench technique" ] } ``` ### OpenAI Chat Model #### Required Schema Each line in your JSONL file must contain a JSON object with these required fields: * **`messages`** (array of objects): The messages in the conversation. * **`role`** (string): The role of the message. * **`content`** (string): The content of the message. #### Example Dataset Entry ```json { "messages": [ { "role": "system", "content": "You are an email writing assistant. Please help people write cogent emails." }, { "role": "user", "content": "Write a concise follow-up after a project review." }, { "role": "assistant", "content": "Thank you for reviewing the project. Please let me know if you have any additional feedback." } ] } ``` ### OpenAI Chat With Tool #### Required Schema Each line in your JSONL file must contain a JSON object with these required fields: * **`messages`** (array of objects): The messages in the conversation, including tool calls. * **`tools`** (array of objects): The available tools that can be called. #### Example Dataset Entry ```json { "messages": [ { "role": "user", "content": "Retrieve information on crimes with no location from the West Yorkshire Police for May 2023 in the categories 'Shoplifting' and 'Violence and Sexual Offences'" }, { "role": "assistant", "content": "", "tool_calls": [ { "type": "function", "function": { "name": "crimes_with_no_location", "arguments": { "date": "2023-05", "force": "West Yorkshire Police", "category": "Shoplifting" } } }, { "type": "function", "function": { "name": "crimes_with_no_location", "arguments": { "date": "2023-05", "force": "West Yorkshire Police", "category": "Violence and Sexual Offences" } } } ] } ], "tools": [ { "type": "function", "function": { "name": "crimes_with_no_location", "description": "Fetches a list of crimes from a specified police force on a given date and category, where the crimes have no specified location.", "parameters": { "type": "object", "properties": { "date": { "description": "The date of the crimes to retrieve in 'YYYY-MM' format.", "type": "string", "default": "2011-09" }, "force": { "description": "The identifier for the police force responsible for handling the crimes.", "type": "string", "default": "warwickshire" }, "category": { "description": "The category of the crimes to retrieve.", "type": "string", "default": "all-crime" } } } } } ] } ``` ### Basic Prompt Completion #### Required Schema Each line in your JSONL file must contain a JSON object with these required fields: * **`prompt`** (string): The input prompt or question. * **`completion`** (string): The expected response or completion. #### Example Dataset Entry ```json { "prompt": "What is the capital of France?", "completion": "Paris." } ``` The `system` plus `conversations: [{from, value}]` layout is **not** a supported SFT schema. Automodel detects chat data from a `messages` array (`role` / `content`), prompt/completion pairs, embedding triplets, or a custom two-column `prompt_template`. Convert conversational examples to [OpenAI Chat Model](#openai-chat-model) format before training; otherwise the job fails at start with `DatasetFormatError`. ### Preference Pairs DPO on NeMo-RL trains on preference pairs. #### Required Schema * **`prompt`** (string, or array of `messages` objects): The prompt. An array can start with a system message. * **`chosen`** (string): The preferred response. * **`rejected`** (string): The dispreferred response. #### Example Dataset Entry ```json { "prompt": "What is the capital of France?", "chosen": "Paris.", "rejected": "I am not sure." } ``` ### Text Unsloth trains on a single text field, named by `dataset.text_field` (`text` by default), as raw text. #### Example Dataset Entry ```json { "text": "Paris is the capital and largest city of France." } ```