Embedding Models

This page provides detailed technical specifications for the embedding model family supported by NeMo Customizer. For information about supported features and capabilities, refer to Tested Models.

Nemotron 3 Embed 1B is the default retrieval checkpoint and trains with training.recipe: bi_encoder. Retrieval quality is measured with retrieve-eval on a frozen eval_beir split, which reports nDCG and Recall rather than the generation metrics used for chat models. For the full pipeline, see Embedding Model Customization and Retrieval SDG.

Llama Nemotron Embed 1B v2 is supported as an alternate checkpoint.

Nemotron 3 Embed 1B

Model Entity Configuration

Register an entity backed by a fileset. A model that Inference Gateway discovered from a running endpoint has fileset: null and cannot be trained.

Training Options

• Full weights (all_weights): 1x 80GB GPU (recommended for the Nemotron recipe)
• LoRA (merged): 1x 80GB GPU, tensor parallel size 1

Resource Requirements

• Minimum GPU memory: 80GB for training; mining ~40GB
• Recommended GPU: A100 or H100
• Side-by-side eval: two GPUs (base NIM + tuned NIM)

Hyperparameter and Data Recommendations

These values match the Nemotron embed recipe. Automodel fills comparable defaults for any field left unset, so set them explicitly when a run has to reproduce the recipe exactly.

Score the fine-tuned model and the base model on the same frozen eval_beir split. Regenerating the split between runs makes the difference reflect data variance as much as model quality.

Deployment Configuration

• NIM image: nvcr.io/nim/nvidia/nemotron-3-embed-1b:2.2.0

• GPU count: 1x 80GB per deployment

• After customization, route through the Inference Gateway:

• Model Entity routing: /apis/inference-gateway/v2/workspaces/{workspace}/model/{name}/-/v1/embeddings

• Provider routing: /apis/inference-gateway/v2/workspaces/{workspace}/provider/{deployment}/-/v1/embeddings

• OpenAI routing: /apis/inference-gateway/v2/workspaces/{workspace}/openai/-/v1/embeddings

import os
from nemo_platform import NeMoPlatform
client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)
response = client.inference.gateway.provider.post(
    "v1/embeddings",
    name="embed-tuned",
    workspace="default",
    body={
        "model": "default/nemotron-3-embed-1b-tuned",
        "input": ["What is the capital of France?"],
        "input_type": "query",
    },
)
print(len(response["data"][0]["embedding"]))

A ModelProvider attaches once the deployment reaches READY. retrieve-eval scores only entities with a non-empty model_providers list.

For the full recipe, from the NVDocs dump through mining, training, deployment, and evaluation, see Embedding Model Customization.

For triplet JSONL schema, see Dataset Format Requirements.

Llama Nemotron Embedding 1B v2

An alternate checkpoint for retrieval.

This checkpoint uses the same Automodel bi_encoder contract; set export.primary to whichever artifact its NIM loads from the fileset root. New work should start from Nemotron 3 Embed 1B on Retriever NIM 2.2.0.