Helm Chart Values for NVIDIA NeMo Retriever Reranking NIM

Use this documentation to learn about the Helm chart parameters for NVIDIA NeMo Retriever Reranking NIM.

Deployment Parameters

Name	Description	Value
affinity	Affinity settings for deployment. Allows to constraint pods to nodes.	{}
containerSecurityContext	Privilege and access control settings for the main container only, for example --set containerSecurityContext.readOnlyRootFilesystem=true.	{}
envVars	Adds arbitrary environment variables to the main container, as a map of names to values, for example --set envVars.NIM_PERFORMANCE_MODE=1. Values are passed as strings. With --set, escape a comma in a value as \,, or use a values file. Don’t also set the env array: if env is set, the chart ignores envVars.	{}
extraVolumes	Adds arbitrary additional volumes to the deployment set definition.	{}
image.repository	The image repository.	""
image.tag	Image tag.	""
image.pullPolicy	Image pull policy.	""
imagePullSecrets	Specify secret names that are needed for the main container and any init containers. Object keys are the names of the secrets.	[{name: ngc-secret}]
nodeSelector	Specify labels to ensure that a product is deployed only on certain nodes (likely best to set this to nvidia.com/gpu.present: "true" depending on cluster setup).	{}
podAnnotations	Specify additional annotation to the main deployment pods.	{}
podSecurityContext	Specify privilege and access control settings for pod (Only affects the main pod).
podSecurityContext.runAsUser	Specify user UID for pod.	1000
podSecurityContext.runAsGroup	Specify group ID for pod.	1000
podSecurityContext.fsGroup	Specify file system owner group id.	1000
replicaCount	Specify replica count for deployment.	1
resources	Specify resources limits and requests for the running service.
resources.limits.nvidia.com/gpu	Specify number of GPUs to present to the running service.	1
serviceAccount.create	Specifies whether a service account should be created.	false
serviceAccount.annotations	Specifies annotations to be added to the service account.	{}
serviceAccount.name	Specify name of the service account to use. If it is not set and create is true, a name is generated using a fullname template.	""
tolerations	Specify tolerations for pod assignment. Allows the scheduler to schedule pods with matching taints.
nim.logLevel	Log level for the main container. The chart passes this value to the container as NIM_LOG_LEVEL.	INFO

Autoscaling Parameters

Values used for autoscaling. If autoscaling is not enabled, these are ignored. They should be overridden on a per-model basis based on quality-of-service metrics as well as cost metrics. This isn’t recommended except with usage of the custom metrics API using something like the prometheus-adapter. Standard metrics of CPU and memory are of limited use in scaling NIM.

Name	Description	Value
autoscaling.enabled	Enable horizontal pod autoscaler.	false
autoscaling.minReplicas	Specify minimum replicas for autoscaling.	1
autoscaling.maxReplicas	Specify maximum replicas for autoscaling.	10
autoscaling.metrics	Array of metrics for autoscaling.	[]

Ingress Parameters

Name	Description	Value
ingress.enabled	Enables ingress.	false
ingress.className	Specify class name for Ingress.	""
ingress.annotations	Specify additional annotations for ingress.	{}
ingress.hosts	Specify list of hosts each containing lists of paths.
ingress.hosts[0].host	Specify name of host.	chart-example.local
ingress.hosts[0].paths[0].path	Specify ingress path.	/
ingress.hosts[0].paths[0].pathType	Specify path type.	ImplementationSpecific
ingress.hosts[0].paths[0].serviceType	Specify service type. It can be nemo or openai – make sure your model serves the appropriate port(s).	openai
ingress.tls	Specify list of pairs of TLS secretName and hosts.	[]

Probe Parameters

Name	Description	Value
livenessProbe.enabled	Enable livenessProbe.	true
livenessProbe.method	Specify the livenessProbe method.	httpGet
livenessProbe.path	LivenessProbe endpoint path.	/v1/health/live
livenessProbe.initialDelaySeconds	Initial delay seconds for livenessProbe.	15
livenessProbe.timeoutSeconds	Timeout seconds for livenessProbe.	1
livenessProbe.periodSeconds	Period seconds for livenessProbe.	10
livenessProbe.successThreshold	Success threshold for livenessProbe.	1
livenessProbe.failureThreshold	Failure threshold for livenessProbe.	3
readinessProbe.enabled	Enable readinessProbe.	true
readinessProbe.method	Specify the readinessProbe method.	httpGet
readinessProbe.path	Readiness Endpoint Path.	/v1/health/ready
readinessProbe.initialDelaySeconds	Initial delay seconds for readinessProbe.	15
readinessProbe.timeoutSeconds	Timeout seconds for readinessProbe.	1
readinessProbe.periodSeconds	Period seconds for readinessProbe.	10
readinessProbe.successThreshold	Success threshold for readinessProbe.	1
readinessProbe.failureThreshold	Failure threshold for readinessProbe.	3
startupProbe.enabled	Enable startupProbe.	true
startupProbe.method	Specify the startupProbe method.	httpGet
startupProbe.path	StartupProbe Endpoint Path.	/v1/health/ready
startupProbe.initialDelaySeconds	Initial delay seconds for startupProbe.	40
startupProbe.timeoutSeconds	Timeout seconds for startupProbe.	1
startupProbe.periodSeconds	Period seconds for startupProbe.	10
startupProbe.successThreshold	Success threshold for startupProbe.	1
startupProbe.failureThreshold	Failure threshold for startupProbe.	180

Storage Parameters

Name	Description	Default Value
persistence	Settings for the persistent volume that is mounted at nim.nimCache (default /opt/nim/.cache).	–
persistence.enabled	True to enable persistent volumes.	false
persistence.existingClaim	An existing PersistentVolumeClaim to use. If you use existingClaim, run only one replica or use a ReadWriteMany storage setup.	""
persistence.storageClass	A persistent volume storage class. If empty, the default provisioner is used.	""
persistence.stsPersistentVolumeClaimRetentionPolicy.whenDeleted	The PVC retention policy when the release is deleted. Only used with StatefulSet volume templates.	Retain
persistence.stsPersistentVolumeClaimRetentionPolicy.whenScaled	The PVC retention policy when the release is scaled. Only used with StatefulSet volume templates.	Retain
persistence.accessMode	The access modes for the storage. If you are using an NFS or similar storage, you can use ReadWriteMany.	ReadWriteOnce
persistence.size	Specify the size of claim, for example 8Gi.	50Gi
hostPath	Configures model cache on local disk on the nodes using hostPath. For special cases only. You should investigate and understand the security implications before you use this option.	""

Service Parameters

Name	Description	Value
service.type	Specify service type for the deployment.	ClusterIP
service.name	Override the default service name.	""
service.httpPort	Specify the HTTP port for the service. Changing it changes only the service port; the container still listens on port 8000.	8000
service.annotations	Specify additional annotations to be added to service.	{}

Reranking Engine Count

The NIM_ENGINE_COUNT environment variable sets the number of GPUs that the NIM uses. For more information, refer to Engine Count.

The chart default is envVars: {}, so NIM_ENGINE_COUNT uses the NIM default of 1. If you set NIM_ENGINE_COUNT to more than 1, also request the same number of GPUs in resources. To set it explicitly, use envVars, as shown in the following example:

envVars:
  NIM_ENGINE_COUNT: "1"

Related Topics

• Deploy NVIDIA NeMo Retriever Reranking NIM on Kubernetes