Is this page helpful?

Helm Chart Values for NVIDIA NeMo Retriever Embedding NIM#

Use this documentation to learn about the Helm chart parameters for NVIDIA NeMo Retriever Embedding NIM.

Note

embedding_type and encoding_format are API request parameters, not Helm chart values. For Nemotron 3.5 Embed 1B, set embedding_type to float, int8, or uint8. Omit it to use the default, float. In release 2.4, binary and ubinary are not supported. The independent encoding_format parameter supports float and base64; base64 encoding does not enable an unsupported embedding type. This output support applies to both BF16 and NVFP4 model profiles and does not change model-weight or inference precision. For details and model-specific support, refer to How to Specify Embedding Type.

Deployment Parameters#

Name

Description

Value

affinity

Affinity settings for deployment. Allows to constraint pods to nodes.

{}

securityContext

Specify privilege and access control settings for Container(Only affects the main container).

{}

envVars

Adds arbitrary environment variables to the main container, as a map of names to values, for example --set envVars.NIM_PERFORMANCE_MODE=1. Values are passed as strings. With --set, escape a comma in a value as \,, or use a values file. Don’t also set the env array: if env is set, the chart ignores envVars.

{}

extraVolumes

Adds arbitrary additional volumes to the deployment set definition.

{}

image.repository

The image repository.

""

image.tag

Image tag.

""

image.pullPolicy

Image pull policy.

""

imagePullSecrets

Specify secret names that are needed for the main container and any init containers. Object keys are the names of the secrets.

[{name: ngc-secret}]

nodeSelector

Specify labels to ensure that a product is deployed only on certain nodes (likely best to set this to nvidia.com/gpu.present: "true" depending on cluster setup).

{}

podAnnotations

Specify additional annotation to the main deployment pods.

{}

podSecurityContext

Specify privilege and access control settings for pod (Only affects the main pod).

podSecurityContext.runAsUser

Specify user UID for pod.

1000

podSecurityContext.runAsGroup

Specify group ID for pod.

1000

podSecurityContext.fsGroup

Specify file system owner group id.

1000

replicaCount

Specify replica count for deployment.

1

resources

Specify resources limits and requests for the running service.

resources.limits.nvidia.com/gpu

Specify number of GPUs to present to the running service.

1

serviceAccount.create

Specifies whether a service account should be created.

false

serviceAccount.annotations

Specifies annotations to be added to the service account.

{}

serviceAccount.name

Specify name of the service account to use. If it is not set and create is true, a name is generated using a fullname template.

""

tolerations

Specify tolerations for pod assignment. Allows the scheduler to schedule pods with matching taints.

logLevel

Log Level to set for the container and metrics collection.

{}

Autoscaling Parameters#

Values used for autoscaling. If autoscaling is not enabled, these are ignored. They should be overridden on a per-model basis based on quality-of-service metrics as well as cost metrics. This isn’t recommended except with usage of the custom metrics API using something like the prometheus-adapter. Standard metrics of CPU and memory are of limited use in scaling NIM.

Name

Description

Value

autoscaling.enabled

Enable horizontal pod autoscaler.

false

autoscaling.minReplicas

Specify minimum replicas for autoscaling.

1

autoscaling.maxReplicas

Specify maximum replicas for autoscaling.

10

autoscaling.metrics

Array of metrics for autoscaling.

[]

Ingress Parameters#

Name

Description

Value

ingress.enabled

Enables ingress.

false

ingress.className

Specify class name for Ingress.

""

ingress.annotations

Specify additional annotations for ingress.

{}

ingress.hosts

Specify list of hosts each containing lists of paths.

ingress.hosts[0].host

Specify name of host.

chart-example.local

ingress.hosts[0].paths[0].path

Specify ingress path.

/

ingress.hosts[0].paths[0].pathType

Specify path type.

ImplementationSpecific

ingress.hosts[0].paths[0].serviceType

Specify service type. It can be nemo or openai – make sure your model serves the appropriate port(s).

openai

ingress.tls

Specify list of pairs of TLS secretName and hosts.

[]

Probe Parameters#

Name

Description

Value

livenessProbe.enabled

Enable livenessProbe.

true

livenessProbe.method

Specify the livenessProbe method.

httpGet

livenessProbe.path

LivenessProbe endpoint path.

/v1/health/live

livenessProbe.initialDelaySeconds

Initial delay seconds for livenessProbe.

15

livenessProbe.timeoutSeconds

Timeout seconds for livenessProbe.

1

livenessProbe.periodSeconds

Period seconds for livenessProbe.

10

livenessProbe.successThreshold

Success threshold for livenessProbe.

1

livenessProbe.failureThreshold

Failure threshold for livenessProbe.

3

readinessProbe.enabled

Enable readinessProbe.

true

readinessProbe.method

Specify the readinessProbe method.

httpGet

readinessProbe.path

Readiness Endpoint Path.

/v1/health/ready

readinessProbe.initialDelaySeconds

Initial delay seconds for readinessProbe.

15

readinessProbe.timeoutSeconds

Timeout seconds for readinessProbe.

1

readinessProbe.periodSeconds

Period seconds for readinessProbe.

10

readinessProbe.successThreshold

Success threshold for readinessProbe.

1

readinessProbe.failureThreshold

Failure threshold for readinessProbe.

3

startupProbe.enabled

Enable startupProbe.

true

startupProbe.method

Specify the startupProbe method.

httpGet

startupProbe.path

StartupProbe Endpoint Path.

/v1/health/ready

startupProbe.initialDelaySeconds

Initial delay seconds for startupProbe.

40

startupProbe.timeoutSeconds

Timeout seconds for startupProbe.

1

startupProbe.periodSeconds

Period seconds for startupProbe.

10

startupProbe.successThreshold

Success threshold for startupProbe.

1

startupProbe.failureThreshold

Failure threshold for startupProbe.

180

Storage Parameters#

Name

Description

Default Value

persistence

Settings for the persistent volume that is mounted at nim.nimCache (default /opt/nim/.cache).

–

persistence.enabled

True to enable persistent volumes.

false

persistence.existingClaim

An existing PersistentVolumeClaim to use. If you use existingClaim, run only one replica or use a ReadWriteMany storage setup.

""

persistence.storageClass

A persistent volume storage class. If empty, the default provisioner is used.

""

persistence.stsPersistentVolumeClaimRetentionPolicy.whenDeleted

The PVC retention policy when the release is deleted. Only used with StatefulSet volume templates.

Retain

persistence.stsPersistentVolumeClaimRetentionPolicy.whenScaled

The PVC retention policy when the release is scaled. Only used with StatefulSet volume templates.

Retain

persistence.accessMode

The access modes for the storage. If you are using an NFS or similar storage, you can use ReadWriteMany.

ReadWriteOnce

persistence.size

Specify the size of claim, for example 8Gi.

50Gi

hostPath

Configures model cache on local disk on the nodes using hostPath. For special cases only. You should investigate and understand the security implications before you use this option.

""

Service Parameters#

Name

Description

Value

service.type

Specify service type for the deployment.

ClusterIP

service.name

Override the default service name.

""

service.httpPort

Specify the HTTP port for the service. Changing it changes only the service port; the container still listens on port 8000. To change the container’s port, refer to the known issues in the release notes.

8000

service.annotations

Specify additional annotations to be added to service.

{}