NeMo Microservices Helm Chart#

Type: application

For deployment guide, see Admin Setup in the NeMo Microservices documentation.

Values#

Key

Type

Default

Description

customizer.customizerConfig.entityStoreURL

string

"http://nemo-entity-store:8000"

Specifies the internal K8s DNS record for the NeMo Entity Store service.

customizer.customizerConfig.mlflowURL

string

""

Specifies the internal K8s DNS record for the MLflow service.

customizer.customizerConfig.models

object

{"meta/llama-3.1-8b-instruct":{"enabled":true}}

Specifies the model information to configure with the NeMo Customizer microservice. To find a complete list of models that NeMo Customizer supports, see Model Catalog.

customizer.customizerConfig.models.”meta/llama-3.1-8b-instruct”.enabled

bool

true

Specifies whether to use the meta/llama-3.1-8b-instruct model for a customization job.

customizer.customizerConfig.nemoDataStoreURL

string

"http://nemo-data-store:3000"

Specifies the internal K8s DNS record for the NeMo Data Store service.

customizer.customizerConfig.training.pvc

object

{"storageClass":""}

Specifies where to store the customization job information during training.

customizer.enabled

bool

true

Specifies whether to install the NeMo Customizer microservice.

customizer.modelsStorage

object

{"accessModes":["ReadWriteOnce"],"enabled":true,"size":"20Gi","storageClassName":""}

Specifies whether to enable the PVC for storing base models.

customizer.modelsStorage.accessModes

list

["ReadWriteOnce"]

Specifies the access modes for the PVC.

customizer.modelsStorage.size

string

"20Gi"

Specifies the size of the PVC.

customizer.modelsStorage.storageClassName

string

""

Specifies the storage class to use for the PVC.

customizer.nemoDataStoreTools

object

{"imagePullSecret":"nvcrimagepullsecret"}

Specifies the image pull secret for the NeMo Data Store tools.

customizer.postgresql.nameOverride

string

"customizerdb"

The name override for the PostgreSQL database.

customizer.postgresql.serviceAccount.create

bool

true

Specifies whether to create a new service account for PostgreSQL.

customizer.postgresql.serviceAccount.name

string

"customizer-postgresql"

customizer.serviceName

string

"nemo-customizer"

customizer.workspaceStorage.enabled

bool

false

Specifies whether to enable the PVC for storing a training job’s workspace.

data-store.enabled

bool

true

Specifies whether to install the NeMo Data Store microservice.

data-store.objectStore.enabled

bool

false

Specifies whether to enable object storage for the NeMo Data Store microservice.

data-store.persistence.accessModes

list

["ReadWriteOnce"]

The access modes for the persistent volume.

data-store.persistence.claimName

string

"datastore-shared-storage"

The name of the persistent volume claim.

data-store.persistence.enabled

bool

true

Specifies whether to enable persistent volume for the NeMo Data Store microservice.

data-store.persistence.size

string

"1Gi"

The size of the persistent volume.

data-store.persistence.storageClass

string

""

The storage class to use for the persistent volume.

data-store.replicaCount

int

1

The number of replicas for the NeMo Data Store microservice deployment.

data-store.serviceAccount.create

bool

true

Specifies whether to create a service account for the NeMo Data Store microservice.

data-store.serviceAccount.name

string

"gitea"

The service account name for the NeMo Data Store microservice.

data-store.serviceName

string

"nemo-data-store"

The service name for the NeMo Data Store microservice.

data-store.strategy.type

string

"Recreate"

The type of deployment strategy to use for the NeMo Data Store microservice.

deployment-management.deployments.entityStoreUrl

string

"http://nemo-entity-store:8000"

The URL for the NeMo Entity Store service.

deployment-management.deployments.modelSyncPeriod

string

"30"

The period in seconds for model synchronization.

deployment-management.deployments.nimImagePullSecrets

list

["nvcrimagepullsecret"]

The image pull secrets for the NeMo Deployment Management microservice.

deployment-management.deployments.nimPeftSource

string

"http://nemo-entity-store:8000"

The URL for the NIM PEFT source.

deployment-management.enabled

bool

true

Specifies whether to install the NeMo Deployment Management microservice.

deployment-management.image.pullPolicy

string

"IfNotPresent"

The pull policy for the NeMo Deployment Management container image.

deployment-management.serviceName

string

"nemo-deployment-management"

The service name for the NeMo Deployment Management microservice.

dgxc-admission-controller.cloudProviderSpec.dummy

object

{"type":"dummy"}

The type of the dummy cloud provider.

dgxc-admission-controller.dgxcController.image

string

"nvcr.io/nvidia/nemo-microservices/dgxc-admission-controller:1.642.0"

The image for the DGX Cloud Admission Controller.

dgxc-admission-controller.enableDGXCRbac

bool

false

dgxc-admission-controller.enableIntegrationsSupport

bool

false

Specifies whether to enable integrations support.

dgxc-admission-controller.enableLabeler

bool

false

Specifies whether to enable the labeler.

dgxc-admission-controller.enableNetworkPolicies

bool

false

Specifies whether to enable network policies.

dgxc-admission-controller.enabled

bool

false

Specifies whether to install the DGX Cloud Admission Controller.

dgxc-admission-controller.imagePullSecret

string

"nvcrimagepullsecret"

The image pull secret for the DGX Cloud Admission Controller.

dgxc-admission-controller.kubernetesAffinity.enabled

bool

false

Specifies whether to enable Kubernetes affinity.

entity-store.appConfig.BASE_URL_DATASTORE

string

"http://nemo-data-store:3000/v1/hf"

The base URL for the NeMo Data Store microservice.

entity-store.appConfig.BASE_URL_NIM

string

"http://nemo-nim-proxy:8000"

The base URL for the NIM service used by the NeMo Entity Store microservice.

entity-store.enabled

bool

true

Specifies whether to install the NeMo Entity Store microservice.

entity-store.postgresql.nameOverride

string

"entity-storedb"

The name override for the PostgreSQL database.

entity-store.postgresql.serviceAccount.create

bool

true

Specifies whether to create a new service account for PostgreSQL.

entity-store.postgresql.serviceAccount.name

string

"entity-store-postgresql"

The service account name for PostgreSQL.

entity-store.serviceName

string

"nemo-entity-store"

The service name for the NeMo Entity Store microservice.

evaluator.argoWorkflows.controller.serviceAccount.name

string

"argo-workflows-controller"

The name of the service account for the Argo Workflows controller.

evaluator.argoWorkflows.crds.install

bool

true

Specifies whether to install the CRDs for Argo Workflows.

evaluator.argoWorkflows.enabled

bool

true

Specifies whether to enable the default Argo Workflows integration for the NeMo Evaluator microservice.

evaluator.argoWorkflows.server.authModes[0]

string

"server"

evaluator.argoWorkflows.server.secure

bool

true

Specifies whether to secure the connection to the Argo Workflows server.

evaluator.argoWorkflows.server.serviceAccount.name

string

"argo-workflows-server"

The name of the service account for the Argo Workflows server.

evaluator.argoWorkflows.server.servicePort

int

2746

The service port for the Argo Workflows server.

evaluator.argoWorkflows.serviceName

string

"argo-workflows-server"

The Argo Workflows service name for the NeMo Evaluator microservice.

evaluator.enabled

bool

true

Specifies whether to install the NeMo Evaluator microservice.

evaluator.external.dataStore

object

{"endpoint":"http://nemo-data-store:3000/v1/hf"}

The external URL of the NeMo Data Store microservice.

evaluator.milvus.standalone.persistence.persistentVolumeClaim.storageClass

string

""

Specifies the storage class to use for the Milvus PVC.

evaluator.postgresql.nameOverride

string

"evaluatordb"

The name override for the default PostgreSQL database.

evaluator.postgresql.primary.networkPolicy.enabled

bool

false

Specifies whether to enable the network policy for the PostgreSQL service.

evaluator.postgresql.serviceAccount.name

string

"evaluator-postgresql"

The name of the service account for PostgreSQL.

evaluator.serviceName

string

"nemo-evaluator"

existingImagePullSecret

string

""

You can specify an existing Kubernetes image pull secret for pulling images from the container registry. The chart doesn’t reference any image pull secret if you leave this empty.

existingSecret

string

""

You can use an existing Kubernetes secret for pulling images. The chart uses the ngcAPIKey value to generate the secret if you leave this empty.

global.imagePullSecrets

list

[{"name":"nvcrimagepullsecret"}]

The name of the image pull secret to use globally across all services.

guardrails.enabled

bool

true

Specifies whether to install the NeMo Guardrails microservice.

guardrails.env

object

{"NIM_ENDPOINT_URL":"http://nemo-nim-proxy:8000/v1"}

The environment variables for the NeMo Guardrails microservice.

guardrails.env.NIM_ENDPOINT_URL

string

"http://nemo-nim-proxy:8000/v1"

The NIM endpoint URL for the NeMo Guardrails microservice.

guardrails.serviceName

string

"nemo-guardrails"

The service name for the NeMo Guardrails microservice.

imagePullSecrets

list

[{"name":"nvcrimagepullsecret","password":"YOUR-NGC-API-KEY","registry":"nvcr.io","username":"$oauthtoken"}]

List of image pull secrets. Existing secrets override these values if you specify them. Use this only for experimentation when you want to hardcode a secret in your values file.

ingress.annotations

object

{}

Annotations for the ingress resource.

ingress.className

string

""

The ingress class to use if your cluster has more than one class.

ingress.enabled

bool

false

Specifies whether to enable the ingress.

ingress.hosts

object

{"dataStore":{"name":"data-store.test","paths":[{"path":"/","pathType":"Prefix","port":3000,"service":"nemo-data-store"}]},"default":{"name":"","paths":[{"path":"/v1/namespaces","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/projects","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/datasets","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/repos","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/customization","pathType":"Prefix","port":8000,"service":"nemo-customizer"},{"path":"/v1/evaluation","pathType":"Prefix","port":7331,"service":"nemo-evaluator"},{"path":"/v1/guardrail","pathType":"Prefix","port":7331,"service":"nemo-guardrails"},{"path":"/v1/deployment","pathType":"Prefix","port":8000,"service":"nemo-deployment-management"}]},"nimProxy":{"name":"nim.test","paths":[{"path":"/v1/completions","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/chat","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"}]}}

A map of hosts and their corresponding paths for the ingress.

ingress.hosts.dataStore.name

string

"data-store.test"

The host name for the third ingress host for the NeMo Data Store microservice.

ingress.hosts.dataStore.paths

list

[{"path":"/","pathType":"Prefix","port":3000,"service":"nemo-data-store"}]

The path rules for the third ingress host.

ingress.hosts.default.name

string

""

The host name for the default ingress host.

ingress.hosts.default.paths

list

[{"path":"/v1/namespaces","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/projects","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/datasets","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/repos","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-entity-store"},{"path":"/v1/customization","pathType":"Prefix","port":8000,"service":"nemo-customizer"},{"path":"/v1/evaluation","pathType":"Prefix","port":7331,"service":"nemo-evaluator"},{"path":"/v1/guardrail","pathType":"Prefix","port":7331,"service":"nemo-guardrails"},{"path":"/v1/deployment","pathType":"Prefix","port":8000,"service":"nemo-deployment-management"}]

The path rules for the default ingress host.

ingress.hosts.nimProxy.name

string

"nim.test"

The host name for the second ingress host for the NIM Proxy microservice.

ingress.hosts.nimProxy.paths

list

[{"path":"/v1/completions","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/chat","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"},{"path":"/v1/models","pathType":"Prefix","port":8000,"service":"nemo-nim-proxy"}]

The path rules for the second ingress host.

ingress.tls

list

[]

TLS configurations.

nemo-operator.controllerManager.manager.scheduler

string

"volcano"

The scheduler to use for the NeMo Operator. Available options: volcano and runai.

nemo-operator.enabled

bool

true

Specifies whether to install the NeMo Operator microservice.

ngcAPIKey

string

"YOUR-NGC-API-KEY"

Your NVIDIA GPU Cloud (NGC) API key authenticates and enables pulling images from the NGC container registry. The existing secret overrides this key if you provide one to the existingSecret key.

nim-operator.enabled

bool

true

Specifies whether to install the NIM Operator.

nim-operator.nfd.nodeFeatureRules.deviceID

bool

false

Specifies whether to enable device ID feature rules.

nim-proxy.enabled

bool

true

Specifies whether to install the NIM Proxy microservice.

nim-proxy.monitoring.enabled

bool

false

Specifies whether to enable monitoring for the NIM Proxy microservice.

nim-proxy.serviceName

string

"nemo-nim-proxy"

The service name for the NIM Proxy microservice.

nim.enabled

bool

false

Specifies whether to deploy a NIM for LLM during the Helm installation of the chart. You can deploy a single static NIM by enabling this object and its values. When enabled, the chart deploys meta/llama-3.1-8b-instruct as the default NIM.

nim.env

list

[{"name":"NIM_PEFT_SOURCE","value":"http://nemo-entity-store:8000"},{"name":"NIM_PEFT_REFRESH_INTERVAL","value":"30"},{"name":"NIM_MAX_CPU_LORAS","value":"16"},{"name":"NIM_MAX_GPU_LORAS","value":"8"}]

Environment variables for the NIM service.

nim.image.repository

string

"nvcr.io/nim/meta/llama-3.1-8b-instruct"

nim.image.tag

string

"1.8.3"

nim.imagePullSecrets[0].name

string

"nvcrimagepullsecret"

nim.model.name

string

"meta/llama-3.1-8b-instruct"

The name of the model to deploy as NIM.

nim.model.ngcAPISecret

string

"ngc-api"

The NGC API secret for model access.

nim.persistence.annotations

object

{"helm.sh/resource-policy":"keep"}

Annotations for the PVC.

nim.persistence.enabled

bool

true

Specifies whether to enable persistence volume claim (PVC) for the NIM service.

nim.persistence.storageClass

string

""

Specifies the storage class to use for the PVC.

nim.resources

object

{"limits":{"nvidia.com/gpu":1},"requests":{"nvidia.com/gpu":1}}

Specifies resources for the NIM service.

nim.resources.requests

object

{"nvidia.com/gpu":1}

Specifies requests for the NIM service.

nim.service.labels

object

{"app.nvidia.com/nim-type":"inference"}

Labels for the NIM service.

nim.statefulSet.enabled

bool

false

Specifies whether to enable a stateful set for the NIM service.

virtualService

object

A virtual service configuration template.

Specifies whether to enable the virtual service. If you are not using istio and virtualservices, it can be useful to create some virtual services for the NeMo Microservices system. Don’t enable unless you use istio.

virtualService.additional.data-store

object

{"entries":{"data-store":{"corsPolicy":{},"match":[{"uri":{"prefix":"/"}}],"route":[{"destination":{"host":"nemo-data-store","port":{"number":3000}}}]}},"gateways":[],"hosts":[]}

Additional virtual service configurations.

virtualService.additional.data-store.entries

object

{"data-store":{"corsPolicy":{},"match":[{"uri":{"prefix":"/"}}],"route":[{"destination":{"host":"nemo-data-store","port":{"number":3000}}}]}}

The entries for the virtual NeMo Data Store service.

virtualService.additional.data-store.entries.data-store.corsPolicy

object

{}

The CORS policy for the virtual NeMo Data Store service.

virtualService.additional.data-store.entries.data-store.match

list

[{"uri":{"prefix":"/"}}]

The match for the virtual NeMo Data Store service.

virtualService.additional.data-store.entries.data-store.route

list

[{"destination":{"host":"nemo-data-store","port":{"number":3000}}}]

The route for the virtual NeMo Data Store service.

virtualService.additional.data-store.gateways

list

[]

The gateways for the virtual NeMo Data Store service.

virtualService.additional.data-store.hosts

list

[]

The hosts for the virtual NeMo Data Store service.

virtualService.additional.nim-proxy.entries.nim-proxy.corsPolicy

object

{}

The CORS policy for the virtual NIM Proxy service.

virtualService.additional.nim-proxy.entries.nim-proxy.match

list

[{"uri":{"prefix":"/"}}]

The match for the virtual NIM Proxy service.

virtualService.additional.nim-proxy.entries.nim-proxy.route

list

[{"destination":{"host":"nemo-nim-proxy","port":{"number":8000}}}]

The route for the virtual NIM Proxy service.

virtualService.additional.nim-proxy.gateways

list

[]

The gateways for the virtual NIM Proxy service.

virtualService.additional.nim-proxy.hosts

list

[]

The hosts for the virtual NIM Proxy service.

virtualService.annotations

object

{}

Annotations for the virtual service.

virtualService.enabled

bool

false

Specifies whether to enable the virtual service.

virtualService.labels

object

{}

Labels for the virtual service.

virtualService.main.entries.customizer.corsPolicy

object

{}

The CORS policy for the virtual NeMo Customizer service.

virtualService.main.entries.customizer.match

list

[{"uri":{"prefix":"/v1/customization"}}]

The match for the virtual NeMo Customizer service.

virtualService.main.entries.customizer.route

list

[{"destination":{"host":"nemo-customizer","port":{"number":8000}}}]

The route for the virtual NeMo Customizer service.

virtualService.main.entries.deployment-management.corsPolicy

object

{}

The CORS policy for the virtual NeMo Deployment Management service.

virtualService.main.entries.deployment-management.match

list

[{"uri":{"prefix":"/v1/deployment"}}]

The match for the virtual NeMo Deployment Management service.

virtualService.main.entries.deployment-management.route

list

[{"destination":{"host":"nemo-deployment-management","port":{"number":8000}}}]

The route for the virtual NeMo Deployment Management service.

virtualService.main.entries.entity-store.corsPolicy

object

{}

The CORS policy for the virtual NeMo Entity Store service.

virtualService.main.entries.entity-store.match

list

[{"uri":{"prefix":"/v1/namespaces"}},{"uri":{"prefix":"/v1/projects"}},{"uri":{"prefix":"/v1/datasets"}},{"uri":{"prefix":"/v1/repos"}},{"uri":{"prefix":"/v1/models"}}]

The match for the virtual NeMo Entity Store service.

virtualService.main.entries.entity-store.route

list

[{"destination":{"host":"nemo-entity-store","port":{"number":8000}}}]

The route for the virtual NeMo Entity Store service.

virtualService.main.entries.evaluator.corsPolicy

object

{}

The CORS policy for the virtual NeMo Evaluator service.

virtualService.main.entries.evaluator.match

list

[{"uri":{"prefix":"/v1/evaluation"}}]

The match for the virtual NeMo Evaluator service.

virtualService.main.entries.evaluator.route

list

[{"destination":{"host":"nemo-evaluator","port":{"number":7331}}}]

The route for the virtual NeMo Evaluator service.

virtualService.main.entries.guardrails.corsPolicy

object

{}

The CORS policy for the virtual NeMo Guardrails service.

virtualService.main.entries.guardrails.match

list

[{"uri":{"prefix":"/v1/guardrail"}}]

The match for the virtual NeMo Guardrails service.

virtualService.main.entries.guardrails.route

list

[{"destination":{"host":"nemo-guardrails","port":{"number":7331}}}]

The route for the virtual NeMo Guardrails service.

virtualService.main.gateways

list

[]

A list of gateways for the virtual service.

virtualService.main.hosts

list

[]

A list of hosts for the virtual service.

volcano.enabled

bool

false

Specifies whether to enable the default Volcano scheduler installation. To learn more, see Volcano.
On this page