Helm Chart Values for NVIDIA NeMo Retriever Embedding NIM#

Use this documentation to learn about the environment variables for NVIDIA NeMo Retriever Embedding NIM.

Deployment Parameters#

Name	Description	Value
`affinity`	Affinity settings for deployment. Allows to constraint pods to nodes.	`{}`
`securityContext`	Specify privilege and access control settings for Container(Only affects the main container).	`{}`
`envVars`	Adds arbitrary environment variables to the main container - Key Value Pairs.	`{}`
`extraVolumes`	Adds arbitrary additional volumes to the deployment set definition.	`{}`
`image.repository`	NIM-LLM Image Repository.	`""`
`image.tag`	Image tag.	`""`
`image.pullPolicy`	Image pull policy.	`""`
`imagePullSecrets`	Specify secret names that are needed for the main container and any init containers. Object keys are the names of the secrets.	`{}`
`nodeSelector`	Specify labels to ensure that NeMo Inference is deployed only on certain nodes (likely best to set this to `nvidia.com/gpu.present: "true"` depending on cluster setup).	`{}`
`podAnnotations`	Specify additional annotation to the main deployment pods.	`{}`
`podSecurityContext`	Specify privilege and access control settings for pod (Only affects the main pod).
`podSecurityContext.runAsUser`	Specify user UID for pod.	`1000`
`podSecurityContext.runAsGroup`	Specify group ID for pod.	`1000`
`podSecurityContext.fsGroup`	Specify file system owner group id.	`1000`
`replicaCount`	Specify replica count for deployment.	`1`
`resources`	Specify resources limits and requests for the running service.
`resources.limits.nvidia.com/gpu`	Specify number of GPUs to present to the running service.	`1`
`serviceAccount.create`	Specifies whether a service account should be created.	`false`
`serviceAccount.annotations`	Specifies annotations to be added to the service account.	`{}`
`serviceAccount.automount`	Specifies whether to automatically mount the service account to the container.	`{}`
`serviceAccount.name`	Specify name of the service account to use. If it is not set and create is true, a name is generated using a fullname template.	`""`
`tolerations`	Specify tolerations for pod assignment. Allows the scheduler to schedule pods with matching taints.

Autoscaling Parameters#

Values used for autoscaling. If autoscaling is not enabled, these are ignored. They should be overridden on a per-model basis based on quality-of-service metrics as well as cost metrics. This isn’t recommended except with usage of the custom metrics API using something like the prometheus-adapter. Standard metrics of CPU and memory are of limited use in scaling NIM.

Name	Description	Value
`autoscaling.enabled`	Enable horizontal pod autoscaler.	`false`
`autoscaling.minReplicas`	Specify minimum replicas for autoscaling.	`1`
`autoscaling.maxReplicas`	Specify maximum replicas for autoscaling.	`10`
`autoscaling.metrics`	Array of metrics for autoscaling.	`[]`

Ingress Parameters#

Name	Description	Value
`ingress.enabled`	Enables ingress.	`false`
`ingress.className`	Specify class name for Ingress.	`""`
`ingress.annotations`	Specify additional annotations for ingress.	`{}`
`ingress.hosts`	Specify list of hosts each containing lists of paths.
`ingress.hosts[0].host`	Specify name of host.	`chart-example.local`
`ingress.hosts[0].paths[0].path`	Specify ingress path.	`/`
`ingress.hosts[0].paths[0].pathType`	Specify path type.	`ImplementationSpecific`
`ingress.hosts[0].paths[0].serviceType`	Specify service type. It can be nemo or openai – make sure your model serves the appropriate port(s).	`openai`
`ingress.tls`	Specify list of pairs of TLS secretName and hosts.	`[]`

Probe Parameters#

Name	Description	Value
`livenessProbe.enabled`	Enable livenessProbe.	`true`
`livenessProbe.method`	LivenessProbe http or script, but no script is currently provided.	`http`
`livenessProbe.path`	LivenessProbe endpoint path.	`/v1/health/live`
`livenessProbe.initialDelaySeconds`	Initial delay seconds for livenessProbe.	`15`
`livenessProbe.timeoutSeconds`	Timeout seconds for livenessProbe.	`1`
`livenessProbe.periodSeconds`	Period seconds for livenessProbe.	`10`
`livenessProbe.successThreshold`	Success threshold for livenessProbe.	`1`
`livenessProbe.failureThreshold`	Failure threshold for livenessProbe.	`3`
`readinessProbe.enabled`	Enable readinessProbe.	`true`
`readinessProbe.path`	Readiness Endpoint Path.	`/v1/health/ready`
`readinessProbe.initialDelaySeconds`	Initial delay seconds for readinessProbe.	`15`
`readinessProbe.timeoutSeconds`	Timeout seconds for readinessProbe.	`1`
`readinessProbe.periodSeconds`	Period seconds for readinessProbe.	`10`
`readinessProbe.successThreshold`	Success threshold for readinessProbe.	`1`
`readinessProbe.failureThreshold`	Failure threshold for readinessProbe.	`3`
`startupProbe.enabled`	Enable startupProbe.	`true`
`startupProbe.path`	StartupProbe Endpoint Path.	`/v1/health/ready`
`startupProbe.initialDelaySeconds`	Initial delay seconds for startupProbe.	`40`
`startupProbe.timeoutSeconds`	Timeout seconds for startupProbe.	`1`
`startupProbe.periodSeconds`	Period seconds for startupProbe.	`10`
`startupProbe.successThreshold`	Success threshold for startupProbe.	`1`
`startupProbe.failureThreshold`	Failure threshold for startupProbe.	`180`

Storage Parameters#

Name	Description	Default Value
`persistence`	Settings to modify the path `/model-store` if `model.legacyCompat` is enabled else `/.cache` volume where the model is served from.	–
`persistence.enabled`	True to enable persistent volumes.	`false`
`persistence.existingClaim`	An existing PersistentVolumeClaim to use. If you use `existingClaim`, run only one replica or use a `ReadWriteMany` storage setup.	`""`
`persistence.storageClass`	A persistent volume storage class. If empty, the default provisioner is used.	`""`
`persistence.stsPersistentVolumeClaimRetentionPolicy.whenDeleted`	The PVC retention policy when the release is deleted. Only used with StatefulSet volume templates.	`Retain`
`persistence.stsPersistentVolumeClaimRetentionPolicy.whenScaled`	The PVC retention policy when the release is scaled. Only used with StatefulSet volume templates.	`Retain`
`persistence.accessMode`	The access modes for the storage. If you are using an NFS or similar storage, you can use `ReadWriteMany`.	`ReadWriteOnce`
`persistence.size`	Specify the size of claim, for example `8Gi`.	`50Gi`
`hostPath`	Configures model cache on local disk on the nodes using hostPath. For special cases only. You should investigate and understand the security implications before you use this option.	`""`

Service Parameters#

Name	Description	Value
`service.type`	Specify service type for the deployment.	`ClusterIP`
`service.name`	Override the default service name.	`""`
`service.http_port`	Specify HTTP Port for the service.	`8080`
`service.annotations`	Specify additional annotations to be added to service.	`{}`

OpenTelemetry Parameters#

Name	Description	Value
`zipkinDeployed`	Specify if this chart should deploy zipkin for metrics.	`false`
`otelDeployed`	Specify if this chart should deploy OpenTelemetry for metrics.	`false`
`otelEnabled`	Specify if this chart should sink metrics to OpenTelemetry.	`false`
`otelEnvVars`	Env variables to configure OTEL in the container, sane defaults in chart.	`{}`
`logLevel`	Log Level to set for the container and metrics collection.	`{}`

OpenTelemetry configurations can be found in the values section of the OpenTelemetry repository.

Note

Configure the OpenTelemetry exporters according to your needs. The provided helm chart provides sample configuration for exporting traces to Zipkin, and metrics to an OTLP-compatible receiver, stored at opentelemetry-collector.config.exporters.zipkin and opentelemetry-collector.config.exporters.otlp, respectively.

For example, if your metrics setup operates in a pull-based fashion and you want to expose NIM metrics in Prometheus format, you can do so by replacing the OTLP exporter with a Prometheus exporter.