> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nemo-helix/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nemo-helix/_mcp/server.

# Embedding Models

This page provides detailed technical specifications for the embedding model family supported by NeMo Customizer. For information about supported features and capabilities, refer to [Tested Models](/documentation/customizer-reference/models/model-catalog).

Nemotron 3 Embed 1B is the default retrieval checkpoint and trains with `training.recipe: bi_encoder`. Retrieval quality is measured with `retrieve-eval` on a frozen `eval_beir` split, which reports nDCG and Recall rather than the generation metrics used for chat models. For the full pipeline, see [Embedding Model Customization](/documentation/customizer-reference/tutorials/embedding-customization-job) and [Retrieval SDG](/documentation/design-synthetic-data/retrieval-sdg).

Llama Nemotron Embed 1B v2 is supported as an alternate checkpoint.

## Nemotron 3 Embed 1B

| Property               | Value                                                                                                                                            |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| Default Name           | nvidia/Nemotron-3-Embed-1B-BF16                                                                                                                  |
| Hugging Face           | [nvidia/Nemotron-3-Embed-1B-BF16](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16)                                                        |
| Training recipe        | `bi_encoder`                                                                                                                                     |
| Embedding dimensions   | 2048                                                                                                                                             |
| Recipe sequence length | 512 (mine, train, and eval)                                                                                                                      |
| Export                 | Set `training.retrieval.export.primary: hf` on the training job. Hugging Face weights land at the fileset root and ONNX under `alternates/onnx`. |
| NIM                    | `nvcr.io/nim/nvidia/nemotron-3-embed-1b:2.2.0` (`/v1/embeddings` with `input_type` query vs document). Use `override_config.nimLegacy: false`.   |
| Finetuning             | `all_weights` or `lora_merged`                                                                                                                   |

### Model Entity Configuration

Register an entity backed by a fileset. A model that Inference Gateway discovered from a running endpoint has `fileset: null` and cannot be trained.

| Configuration   | Value                                          |
| --------------- | ---------------------------------------------- |
| Workspace       | default                                        |
| Name            | nemotron-3-embed-1b                            |
| Fileset storage | Hugging Face `nvidia/Nemotron-3-Embed-1B-BF16` |
| Precision       | bf16                                           |

### Training Options

* **Full weights (`all_weights`)**: 1x 80GB GPU (recommended for the Nemotron recipe)
* **LoRA (merged)**: 1x 80GB GPU, tensor parallel size 1

Retriever NIM 2.2.0 serves a full checkpoint, not a standalone adapter, so `training.finetuning_type` has to be `all_weights` or `lora_merged`. Unmerged `lora` is rejected for the `bi_encoder` recipe.

### Resource Requirements

* **Minimum GPU memory**: 80GB for training; mining \~40GB
* **Recommended GPU**: A100 or H100
* **Side-by-side eval**: two GPUs (base NIM + tuned NIM)

### Hyperparameter and Data Recommendations

These values match the Nemotron embed recipe. Automodel fills comparable defaults for any field left unset, so set them explicitly when a run has to reproduce the recipe exactly.

| Parameter       | API Field Name                                                                             | Recommended Value                                                      |
| --------------- | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------- |
| Learning rate   | `optimizer.learning_rate`                                                                  | `1e-5` (sweep `5e-6` / `2e-5`)                                         |
| Warmup          | `optimizer.warmup_steps`                                                                   | `5`                                                                    |
| Weight decay    | `optimizer.weight_decay`                                                                   | `0.01`                                                                 |
| Epochs          | `schedule.epochs`                                                                          | `1`–`2` on NVDocs-scale or real corpora (`3` is for tiny example sets) |
| Global batch    | `batch.global_batch_size`                                                                  | `128`                                                                  |
| Micro batch     | `batch.micro_batch_size`                                                                   | `4`                                                                    |
| Sequence length | `training.max_seq_length` / `training.retrieval.query_max_length` and `passage_max_length` | `512`                                                                  |
| Passages        | `training.retrieval.train_n_passages`                                                      | `5` (1 positive + 4 hard negatives)                                    |

Score the fine-tuned model and the base model on the same frozen `eval_beir` split. Regenerating the split between runs makes the difference reflect data variance as much as model quality.

### Deployment Configuration

* NIM image: `nvcr.io/nim/nvidia/nemotron-3-embed-1b:2.2.0`

* GPU count: 1x 80GB per deployment

* After customization, route through the **Inference Gateway**:

* **Model Entity routing**: `/apis/inference-gateway/v2/workspaces/{workspace}/model/{name}/-/v1/embeddings`

* **Provider routing**: `/apis/inference-gateway/v2/workspaces/{workspace}/provider/{deployment}/-/v1/embeddings`

* **OpenAI routing**: `/apis/inference-gateway/v2/workspaces/{workspace}/openai/-/v1/embeddings`

```python
import os
from nemo_platform import NeMoPlatform

client = NeMoPlatform(
    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
    workspace="default",
)

response = client.inference.gateway.provider.post(
    "v1/embeddings",
    name="embed-tuned",
    workspace="default",
    body={
        "model": "default/nemotron-3-embed-1b-tuned",
        "input": ["What is the capital of France?"],
        "input_type": "query",
    },
)
print(len(response["data"][0]["embedding"]))
```

A ModelProvider attaches once the deployment reaches `READY`. `retrieve-eval` scores only entities with a non-empty `model_providers` list.

For the full recipe, from the NVDocs dump through mining, training, deployment, and evaluation, see [Embedding Model Customization](/documentation/customizer-reference/tutorials/embedding-customization-job).

For triplet JSONL schema, see [Dataset Format Requirements](/documentation/customizer-reference/models/dataset-format).

## Llama Nemotron Embedding 1B v2

An alternate checkpoint for retrieval.

| Property             | Value                                                                                                                                                                                       |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Default Name         | nvidia/llama-nemotron-embed-1b-v2                                                                                                                                                           |
| Hugging Face         | [nvidia/llama-nemotron-embed-1b-v2](https://huggingface.co/nvidia/llama-nemotron-embed-1b-v2)                                                                                               |
| NIM                  | [nvidia/llama-nemotron-embed-1b-v2](https://catalog.ngc.nvidia.com/orgs/nim/teams/nvidia/containers/llama-nemotron-embed-1b-v2) (`1.13.0` is a 1.x image; leave `nimLegacy` at the default) |
| Max sequence length  | 8192                                                                                                                                                                                        |
| Embedding dimensions | 2048 (Matryoshka: 384, 512, 768, 1024, or 2048)                                                                                                                                             |
| Finetuning           | `all_weights` or `lora_merged`                                                                                                                                                              |

This checkpoint uses the same Automodel `bi_encoder` contract; set `export.primary` to whichever artifact its NIM loads from the fileset root. New work should start from Nemotron 3 Embed 1B on Retriever NIM 2.2.0.