API Reference (gRPC) for NVIDIA NeMo Retriever Embedding NIM#

This documentation contains the gRPC reference for NVIDIA NeMo Retriever Embedding NIM.

gRPC Models#

The gRPC model names differ from the NIM model IDs shown in the Support Matrix. The following table contains the mapping of the names.

Model ID	gRPC Model Name
nvidia/Nemotron-3.5-Embed-1B-BF16	nvidia_nemotron_3_5_embed_1b_bf16
nvidia/Nemotron-3.5-Embed-1B-NVFP4	nvidia_nemotron_3_5_embed_1b_nvfp4
nvidia/nemotron-3-embed-1b	nvidia_nemotron_3_embed_1b
nvidia/llama-nemotron-embed-1b-v2	nvidia_llama_nemotron_embed_1b_v2
nvidia/nv-embedqa-e5-v5	nvidia_nv_embedqa_e5_v5

Both finalized Nemotron 3.5 Embed models support unary gRPC inference through the KServe V2 protocol. Enable and publish the gRPC listener as described in Launch the NIM. The table lists the default served names. If you configure a custom served model name, use its lowercase form with each character other than an ASCII letter or digit replaced by an underscore. Query model metadata to confirm the served gRPC name.

Request Inputs#

Input	Shape	Data Type	Description	Required
`text`	[batch_size, 1]	BYTES	A list of UTF-8 encoded strings to embed. For details on how to encode multimodal data as string, refer to How to Specify Modality.	Yes
`modality`	[batch_size, 1]	BYTES	A list of UTF-8 modality strings for each of the text input elements. If you don’t specify modality, the modality is inferred. For supported modalities, refer to How to Specify Modality.	No

Request Parameters#

Parameter	Data Type	Description	Valid Values	Default	Required
`input_type`	String	The context of the embedding.	`"query"`, `"passage"`	`"query"`	Yes
`truncate`	String	How to handle text that exceeds the deployment’s maximum token length. See Nemotron 3.5 truncation.	Nemotron 3.5: `"END"`, `"NONE"`. Other models: `"END"`, `"START"`, `"NONE"`.	Nemotron 3.5: `"END"`. Other models: `"NONE"`.	No for Nemotron 3.5; yes for other models.
`dimensions`	Integer	The desired dimensionality of the output embeddings. See Nemotron 3.5 dimensions. For `nvidia/nemotron-3-embed-1b`, reduced dimensions are not supported.	Nemotron 3.5: any integer from `128` through `2048`. `nvidia/nemotron-3-embed-1b`: `2048` only.	The model’s native dimension; `2048` for these models.	No
`embedding_type`	String	The output type of the embeddings. See How to Specify Embedding Type for how the output type is handled. For Nemotron 3.5, `"binary"` and `"ubinary"` return gRPC `INVALID_ARGUMENT` before inference, including requests that use a custom served model name.	Nemotron 3.5: `"float"`, `"int8"`, `"uint8"`. Other models: `"float"`, `"binary"`, `"ubinary"`, `"int8"`, `"uint8"`.	`"float"`	No
`nvcf_asset_dir`	String	Directory path where NVCF (NVIDIA Cloud Functions) asset files are stored. See API Reference (OpenAPI) for NVIDIA NeMo Retriever Embedding NIM for more details.	-	-	No

Nemotron 3.5 Truncation#

The following rules apply to both nvidia/Nemotron-3.5-Embed-1B-BF16 and nvidia/Nemotron-3.5-Embed-1B-NVFP4:

END removes text from the end when the input exceeds the deployment’s sequence limit.
NONE rejects input that exceeds that limit.
Omitting truncate uses END.
START is not supported. The server returns gRPC INVALID_ARGUMENT before inference for both text and image-bearing requests.

For image-bearing requests, truncation does not remove the image block or passage prefix. The server rejects the request if those required tokens cannot fit within the sequence limit. These rules do not change the truncation contract of other models on this page.

Nemotron 3.5 Dimensions#

Both finalized Nemotron 3.5 Embed models accept any integer dimensions value from 128 through 2048, inclusive. The values are not limited to a fixed set. Omitting the parameter returns the native 2048 dimensions. For a reduced dimension, the server takes the first requested number of vector elements and applies L2 normalization again. Values outside this range return gRPC INVALID_ARGUMENT before inference. Other models retain their model-specific dimension constraints. See model-specific dimension constraints for the shared HTTP dimension documentation.

Response#

Output	Shape	Data Type	Description
`token_count`	[batch_size]	INT32	The number of tokens in each input text.
`embeddings`	[batch_size, embedding_dimension]	Configurable using the `embedding_type` request parameter. The default is FLOAT32.	The resulting embedding vectors.