Dataset Format Requirements

Use the following guidelines to prepare your training dataset for the supported models.

Dataset Preparation Guidelines

• File Format: Save your training data as .jsonl files (one JSON object per line).
• Validation: Each record is automatically validated against the appropriate schema when training begins. The required format depends on training.training_type (for example, sft) specified in your job configuration.

For dataset creation tutorials, refer to Format Training Dataset.

Prepare Data Designer Output or a Table

nemo customization dataset prepare turns a Parquet, CSV or JSONL table, such as Data Designer output, into a training dataset in the format you name with --format. It maps the columns, splits the rows with a seeded shuffle, and uploads the splits to a new dataset fileset, printing the references to put in your job:

# A local file or directory, or a fileset
nemo customization dataset prepare ./rows.parquet --name sft-data --format chat
nemo customization dataset prepare default/my-table --name dpo-data --format preference

To prepare the output of a Data Designer job, download its artifacts result and pass the parquet-files directory inside it:

nemo jobs results download artifacts --job my-dd-job --workspace default -o artifacts.tar
tar -xf artifacts.tar
nemo customization dataset prepare "$(find . -type d -name parquet-files)" --name sft-data --format chat

The command does not prepare GRPO datasets: GRPO trains on NeMo Gym rows that each environment builds.

The Studio “Instruction fine-tuning” and “Preference pairs” templates need no column flags. For other column names, pass the flag for the field: --prompt-column, --response-column, --system-column (adds a system message, for chat and preference; with it, every preference prompt is a message list, so the column has one type), --messages-column, --chosen-column, --rejected-column, --query-column, --positive-column, --negatives-column or --text-column. A flag the format does not use, or a column the table does not have, fails before anything is uploaded. --input-format reads files whose extension does not give their format, or one format from a directory that mixes them.

Every format writes train.jsonl and validation.jsonl. With --val-fraction 0 there is no validation.jsonl, and training holds out its own validation rows. The test split (--test-fraction, 0 by default for preference and 0.1 otherwise) goes to a separate <name>-test fileset, so training never reads it. --val-fraction (default 0.1) and --seed (default 42) set the rest; the same seed always gives the same split, and a fraction above 0 gets at least one row even from a small table. Rows with an empty required field, such as a failed generation, are dropped and counted; an empty system prompt is left out and empty negative documents are skipped.

Dataset Formats

The following sections describe the required schema and example dataset entries for each model type.

Embedding Model

For embedding model training, the following additional guidelines apply:

• Negative Examples: Provide multiple negative documents per query for better contrastive learning.
• Content Quality: Ensure positive documents are semantically relevant to the query.
• JSONL Format: The supported embedding models require training data in JSONL format with a specific triplet structure for contrastive learning.

Required Schema

Each line in your JSONL file must contain a JSON object with these required fields:

• query (string): The query text to use as an anchor for similarity learning.
• pos_doc (string): A document that should match positively with the query.
• neg_doc (array of strings): One or more documents that should not match with the query. Retrieval Stage 1 run without mining (enable_mining: false) emits an empty list. Automodel bi_encoder / cross_encoder jobs then fail at collate time with neg_doc must contain at least 1 document to sample N negatives (N defaults to 4). Mine hard negatives before submitting, or provide your own non-empty lists. Do not lower training.retrieval.train_n_passages to hide an unmined fileset.

Example Dataset Entry

{
  "query": "3D trajectory recovery for tracking multiple objects",
  "pos_doc": "Recursive Estimation of Motion, Structure, and Focal Length",
  "neg_doc": [
    "Characterization of 1.2 kV SiC super-junction SBD implemented by trench technique"
  ]
}

OpenAI Chat Model

Required Schema

Each line in your JSONL file must contain a JSON object with these required fields:

• messages (array of objects): The messages in the conversation.
  • role (string): The role of the message.
  • content (string): The content of the message.

Example Dataset Entry

{
  "messages": [
    {
      "role": "system",
      "content": "You are an email writing assistant. Please help people write cogent emails."
    },
    {
      "role": "user",
      "content": "Write a concise follow-up after a project review."
    },
    {
      "role": "assistant",
      "content": "Thank you for reviewing the project. Please let me know if you have any additional feedback."
    }
  ]
}

OpenAI Chat With Tool

Required Schema

Each line in your JSONL file must contain a JSON object with these required fields:

• messages (array of objects): The messages in the conversation, including tool calls.
• tools (array of objects): The available tools that can be called.

Example Dataset Entry

{
  "messages": [
    {
      "role": "user",
      "content": "Retrieve information on crimes with no location from the West Yorkshire Police for May 2023 in the categories 'Shoplifting' and 'Violence and Sexual Offences'"
    },
    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "type": "function",
          "function": {
            "name": "crimes_with_no_location",
            "arguments": {
              "date": "2023-05",
              "force": "West Yorkshire Police",
              "category": "Shoplifting"
            }
          }
        },
        {
          "type": "function",
          "function": {
            "name": "crimes_with_no_location",
            "arguments": {
              "date": "2023-05",
              "force": "West Yorkshire Police",
              "category": "Violence and Sexual Offences"
            }
          }
        }
      ]
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "crimes_with_no_location",
        "description": "Fetches a list of crimes from a specified police force on a given date and category, where the crimes have no specified location.",
        "parameters": {
          "type": "object",
          "properties": {
            "date": {
              "description": "The date of the crimes to retrieve in 'YYYY-MM' format.",
              "type": "string",
              "default": "2011-09"
            },
            "force": {
              "description": "The identifier for the police force responsible for handling the crimes.",
              "type": "string",
              "default": "warwickshire"
            },
            "category": {
              "description": "The category of the crimes to retrieve.",
              "type": "string",
              "default": "all-crime"
            }
          }
        }
      }
    }
  ]
}

Basic Prompt Completion

Required Schema

Each line in your JSONL file must contain a JSON object with these required fields:

• prompt (string): The input prompt or question.
• completion (string): The expected response or completion.

Example Dataset Entry

{
  "prompt": "What is the capital of France?",
  "completion": "Paris."
}

The system plus conversations: [{from, value}] layout is not a supported SFT schema. Automodel detects chat data from a messages array (role / content), prompt/completion pairs, embedding triplets, or a custom two-column prompt_template. Convert conversational examples to OpenAI Chat Model format before training; otherwise the job fails at start with DatasetFormatError.

Preference Pairs

DPO on NeMo-RL trains on preference pairs.

Required Schema

• prompt (string, or array of messages objects): The prompt. An array can start with a system message.
• chosen (string): The preferred response.
• rejected (string): The dispreferred response.

Example Dataset Entry

{
  "prompt": "What is the capital of France?",
  "chosen": "Paris.",
  "rejected": "I am not sure."
}

Text

Unsloth trains on a single text field, named by dataset.text_field (text by default), as raw text.

Example Dataset Entry

{
  "text": "Paris is the capital and largest city of France."
}