Format Training Dataset

Learn how to format a training dataset to work with the model type you want to train, such as a chat or completion model.

• Chat Model — Requires data that adheres to the messages schema.
• Completion Model — Requires data that adheres to the prompt-completion schema.

Customizer expects all datasets to use JSONL format, where each line in the dataset is a training example serialized in JSON.

Prerequisites

Dataset Best Practices

Before formatting your dataset, follow these principles to ensure high-quality training:

Data Quality Checklist

Before uploading your dataset, verify:

✅ Format: All entries are valid JSON on a single line (JSONL) ✅ Schema: Required fields present (messages, prompt/completion, or custom columns) ✅ Encoding: UTF-8 encoding (not UTF-16 or other encodings) ✅ Completeness: No empty or null values in required fields ✅ Length: Examples fit within model’s context window (typically 2048-8192 tokens) ✅ Split: Training and validation sets are separate and representative

Common Dataset Issues

Overfitting symptoms:

• Training loss → 0 but validation loss stays high or increases
• Model memorizes training examples verbatim
• Poor generalization to new inputs

Solutions:

• Increase dataset size and diversity
• Reduce training epochs (try 1-3 instead of 5+)
• Add more validation data
• Use regularization (dropout, lower learning rate)

Underfitting symptoms:

• Both training and validation loss remain high
• Model output quality is poor even on training examples
• Loss plateaus early in training

Solutions:

• Increase training epochs (try 5-10 instead of 3)
• Increase model size if using small models
• Check data quality and consistency
• Adjust learning rate (try 5e-5 to 1e-4)

Chat Models

Chat and instruction models require additional structure in their data to capture concepts such as multi-turn conversations. We support the OpenAI messages format.

Format a Conversation Dataset

Train a chat model to optimize for generating responses using multiple messages as context.

Basic Schema

A conversational dataset contains a sequence of messages that represent interactions between users and assistants. Each message has:

• A role field to categorize the message text. Options include system, user, and assistant.
• A content field for the actual body of information communicated by that role.

Example entry formatted for JSONL dataset file:

1
{
2
  "messages": [
3
    {
4
      "role": "system",
5
      "content": "<system message>"
6
    }, {
7
      "role": "user",
8
      "content": "<user message>"
9
    }, {
10
      "role": "assistant",
11
      "content": "<assistant message>"
12
    }
13
  ]
14
}

Reasoning Considerations

Some models (such as Llama Nemotron) support a detailed thinking mode, which you can toggle in the system message. This setting controls whether the model is encouraged to show step-by-step reasoning in its responses.

• Training data without reasoning: Use detailed thinking off in the system message.
• Training data with reasoning: Use detailed thinking on in the system message.

1
{"messages": [
2
  {"role": "system", "content": "detailed thinking off"},
3
  {"role": "user", "content": "What is 2 + 2?"},
4
  {"role": "assistant", "content": "4"}
5
]}

You can adjust the system message for each training example to match the style of your data and the behavior you want the model to learn.

Schema with Tool Calling

Tool calling (also known as function calling) allows the model to directly interact with external systems based on user inputs. By integrating the model with external applications and APIs, you can significantly expand its capabilities to include:

• Data retrieval from databases or services
• Execution of specific actions in connected systems
• Performance of computation tasks
• Implementation of complex business logic

Inline Tools

To train a model with tool calling capabilities, use the conversational dataset format with additional fields. Beyond the standard messages structure, you’ll need to define the list of tools the model can access along with their function parameters. Within the messages, include tool_calls when you want the model to invoke a tool with specific arguments.

Every sample must be a single line as in this example below.

1
{
2
  "messages": [
3
    {
4
      "role": "user",
5
      "content": ""
6
    },
7
    {
8
      "role": "assistant",
9
      "content": "",
10
      "tool_calls": [{
11
        "type": "function",
12
        "function": {
13
          "name": "fibonacci",
14
          "arguments": {"n": 20}
15
        }
16
      }]
17
    }
18
  ],
19
  "tools": [{
20
    "type": "function",
21
    "function": {
22
      "name": "fibonacci",
23
      "description": "Calculates the nth Fibonacci number.",
24
      "parameters": {
25
        "type": "object",
26
        "properties": {
27
          "n": {
28
            "description": "The position of the Fibonacci number.",
29
            "type": "integer"
30
          }
31
        }
32
      }
33
    }
34
  }]
35
}

Shared Tools

When your dataset uses the same set of tools across all examples, you can streamline your configuration by omitting the tools field from individual dataset entries. Instead, specify these tools once in the dataset_parameters section during job creation.

1
import os
2
from nemo_platform import NeMoPlatform
3
4
client = NeMoPlatform(
5
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
6
    workspace="default",
7
)
8
9
# Create a customization job with shared tools
10
job = client.customization.jobs.create(
11
    name="my-tool-calling-job",
12
    workspace="default",
13
    spec={
14
        "model": "default/llama-3-2-1b",
15
        "dataset": "fileset://default/my-tool-dataset",
16
        "dataset_parameters": {
17
            "tools": [
18
                {
19
                    "type": "function",
20
                    "function": {
21
                        "name": "fibonacci",
22
                        "description": "Calculates the nth Fibonacci number.",
23
                        "parameters": {
24
                            "type": "object",
25
                            "properties": {
26
                                "n": {
27
                                    "description": "The position of the Fibonacci number.",
28
                                    "type": "integer",
29
                                }
30
                            },
31
                        },
32
                    },
33
                }
34
            ]
35
        },
36
        "training": {
37
            "type": "sft",
38
            "peft": {"type": "lora", "rank": 8, "alpha": 32},
39
            "epochs": 10,
40
            "batch_size": 16,
41
            "learning_rate": 1e-4,
42
        },
43
    },
44
)
45
46
print(f"Created job: {job.name}")

Find Chat Models

You can retrieve a Model Entity to check if it’s a chat model by examining the spec.is_chat field:

1
import os
2
from nemo_platform import NeMoPlatform
3
4
client = NeMoPlatform(
5
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
6
    workspace="default",
7
)
8
9
# Get a specific model
10
model = client.models.retrieve(name="llama-3-2-1b", workspace="default")
11
12
print(f"Model: {model.name}")
13
if model.spec:
14
    print(f"Is Chat Model: {model.spec.is_chat}")
15
    print(f"Family: {model.spec.family}")
16
    print(f"Parameters: {model.spec.base_num_parameters:,}")
17
    print(f"Max Sequence Length: {model.spec.max_sequence_length}")

Chat with the Model

To run inference with a chat model, you first need to deploy the model, then use the Inference Gateway to send requests.

1
import os
2
from nemo_platform import NeMoPlatform
3
4
client = NeMoPlatform(
5
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
6
    workspace="default",
7
)
8
9
# Use Inference Gateway with OpenAI-compatible client
10
oai_client = client.models.get_openai_client(workspace="default")
11
12
# Create a chat completion (model format: "workspace/model-entity-name")
13
response = oai_client.chat.completions.create(
14
    model="default/llama-3-2-1b",
15
    messages=[
16
        {"role": "system", "content": "You are a helpful assistant."},
17
        {"role": "user", "content": "Hello! How are you?"},
18
    ],
19
    temperature=0.5,
20
    top_p=1,
21
    max_tokens=1024,
22
)
23
24
print(f"Response: {response.choices[0].message.content}")

Next Steps

Now that you know how to format your training datasets, you can proceed with creating customization jobs:

• Start a LoRA Model Customization Job - For parameter-efficient fine-tuning
• Start a Full SFT Customization Job - For full model fine-tuning

Completion Models

Train a model using the completion dataset format for tasks like text summarization, information extraction, question answering, text classification, reasoning, or story writing.

Format a Prompt-Completion Dataset

Prompt completion datasets have a simple schema. Each datum has:

• A prompt field for the body of information provided by the user.
• A completion field for output of the model.

Prompt the Model

To run inference with a completion model, use the Inference Gateway to send requests to the /completions endpoint.

1
import os
2
from nemo_platform import NeMoPlatform
3
4
client = NeMoPlatform(
5
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
6
    workspace="default",
7
)
8
9
# Use Inference Gateway with OpenAI-compatible client
10
oai_client = client.models.get_openai_client(workspace="default")
11
12
# Create a completion (model format: "workspace/model-entity-name")
13
response = oai_client.completions.create(
14
    model="default/llama-3-2-1b",
15
    prompt="Once upon a time",
16
    temperature=0.5,
17
    top_p=1,
18
    max_tokens=1024,
19
)
20
21
print(f"Response: {response.choices[0].text}")