Embedding Models

This page provides detailed technical specifications for the embedding model family supported by NeMo Customizer. For information about supported features and capabilities, refer to Tested Models.

Nemotron 3 Embed 1B is the default retrieval checkpoint and trains with training.recipe: bi_encoder. Retrieval quality is measured with retrieve-eval on a frozen eval_beir split, which reports nDCG and Recall rather than the generation metrics used for chat models. For the full pipeline, see Embedding Model Customization and Retrieval SDG.

Llama Nemotron Embed 1B v2 is supported as an alternate checkpoint.

Nemotron 3 Embed 1B

Model Entity Configuration

Register an entity backed by a fileset. A model that Inference Gateway discovered from a running endpoint has fileset: null and cannot be trained.

Training Options

• Full weights (all_weights): 4x 80GB GPU, tensor parallel size 1 (validated NVDocs recipe)
• LoRA (merged): 1x 80GB GPU, tensor parallel size 1

Resource Requirements

• Minimum GPU memory: 80GB for training; mining ~40GB
• Recommended GPU: A100 or H100
• Side-by-side eval: two GPUs (base NIM + tuned NIM)

Hyperparameter and Data Recommendations

These values are the configuration validated on the frozen 20,909-query NVDocs eval_beir split.

On that split, the untuned base scores nDCG@10 0.5671, Recall@10 0.6375, and Recall@100 0.8238. The recipe above scores nDCG@10 0.6318, Recall@10 0.7124, and Recall@100 0.8958 (uplift +0.0647 / +11.4% nDCG@10, +0.0748 / +11.7% Recall@10, +0.0720 / +8.7% Recall@100). Validation loss was lowest at the final step, so the best and last checkpoints match. MAP@10 moves from 0.4787 to 0.5413.

Validation data is optional. Automodel first looks in the dataset.training fileset for files named val*.jsonl, validation*.jsonl, or dev*.jsonl. If none exists, schedule.validation_split deterministically holds out part of the discovered training rows. This split drives validation loss and checkpoint selection. It is separate from the frozen eval_beir split used by retrieve-eval to compare retrieval quality.

Score the fine-tuned model and the base model on the same frozen eval_beir split. Regenerating the split between runs makes the difference reflect data variance as much as model quality.

Deployment Configuration

• NIM image: nvcr.io/nim/nvidia/nemotron-3-embed-1b:2.2.0

• GPU count: 1x 80GB per deployment

• After customization, route through the Inference Gateway:

• Model Entity routing: /apis/inference-gateway/v2/workspaces/{workspace}/model/{name}/-/v1/embeddings

• Provider routing: /apis/inference-gateway/v2/workspaces/{workspace}/provider/{deployment}/-/v1/embeddings

• OpenAI routing: /apis/inference-gateway/v2/workspaces/{workspace}/openai/-/v1/embeddings

import os
from nemo_helix import NeMoHelix
client = NeMoHelix(
    base_url=os.environ.get("NHX_BASE_URL", "http://localhost:8080"),
    workspace="default",
)
response = client.inference.gateway.provider.post(
    "v1/embeddings",
    name="embed-tuned",
    workspace="default",
    body={
        "model": "default/nemotron-3-embed-1b-tuned",
        "input": ["What is the capital of France?"],
        "input_type": "query",
    },
)
print(len(response["data"][0]["embedding"]))

A ModelProvider attaches once the deployment reaches READY. retrieve-eval scores only entities with a non-empty model_providers list.

For the full recipe, from the NVDocs dump through mining, training, deployment, and evaluation, see Embedding Model Customization.

For triplet JSONL schema, see Dataset Format Requirements.

Llama Nemotron Embedding 1B v2

An alternate checkpoint for retrieval.

This checkpoint uses the same Automodel bi_encoder contract; set export.primary to whichever artifact its NIM loads from the fileset root. New work should start from Nemotron 3 Embed 1B on Retriever NIM 2.2.0.