Ingress Setup for Production Environment

Most Kubernetes clusters include an in-cluster network that is encapsulated and can typically initiate outbound connections (egress) unless controlled by specific policies. Incoming traffic (ingress) from the external or physical network must typically be routed through a LoadBalancer or NodePort service type.

In bare metal environments, where the cluster network is extended or peered to the top-of-rack switches and is therefore contiguous with the physical network, using an ingress controller or a similar proxy solution is still advisable. With this approach, you manage routing for the services.

Because the NeMo microservices operate at Layer 7 (HTTP-based APIs), implementing a fully HTTP-aware controller—such as an ingress controller or gateway—enables proper routing for building a cohesive system.

The NeMo Microservices Helm Chart comes with default values to route incoming requests appropriately. You can refer to the ingress object in the values.yaml file for setting up an ingress resource, which you can use with most ingress controllers. It also provides a virtual service template for cases where you use an Istio Gateway to handle ingress.

Domain Names

Ensure that you define DNS records that will work in your Kubernetes cluster, ideally with TLS certificates.

You need at least three DNS records for the following:

1. One for a default host for most of the NeMo microservices. The default path rules for the default host:

   Path (Prefix)	Service	Port Number
   /v1/namespaces	nemo-entity-store	8000
   /v1/projects	nemo-entity-store	8000
   /v1/datasets	nemo-entity-store	8000
   /v1/repos	nemo-entity-store	8000
   /v1/models	nemo-entity-store	8000
   /v1/customization	nemo-customizer	8000
   /v1/evaluation	nemo-evaluator	7331
   /v1/guardrail	nemo-guardrails	7331
   /v1/deployment	nemo-deployment-management	8000

2. One for NIM Proxy. The default path rules for nimProxy:

   Path (Prefix)	Service	Port Number
   /v1/completions	nemo-nim-proxy	8000
   /v1/chat	nemo-nim-proxy	8000
   /v1/models	nemo-nim-proxy	8000

3. One for NeMo Data Store. The default path rule for dataStore:

   Path (Prefix)	Service	Port Number
   /	nemo-data-store	3000

   For a complete list of the network ports for the NeMo microservices, refer to the Security for NeMo Microservices page.

For example, assume that you create a domain name called nemo.example.com for the default host, nim.example.com for nimProxy, and datastore.example.com for dataStore. Create these domain names in your DNS provider. If you use TLS certificates, refer to the documentation from your ingress controller provider about how to correctly configure them in the cluster.

Note

Specify the default host name in the values file only if you already have other ingressed systems in your cluster. Otherwise, it functions as the default destination for requests.

In your custom values file for the NeMo Microservices Helm Chart installation, configure the ingress object as follows:

ingress:
  enabled: true
  tls: [] #if you have tls configs, enter them here
  hosts:
    default:
      name: nemo.example.com
    nimProxy:
      name: nim.example.com
    dataStore:
      name: datastore.example.com

As long as your ingress controller supports Prefix path types, you might not need to adjust the paths under these hosts. Run helm show values nemo-microservices-helm-chart-25.4.0.tgz to view the default ingress object and check if anything doesn’t work. If any default paths entries need to change in such cases where you overwrite the service names for the NeMo microservices, make sure that you re-enter all the correct paths for the corresponding host in the ingress values.

Annotations

Use the ingress.annotations value to add any of the annotations you require. Most advanced functionality for ingress controllers is managed via annotations.

Authentication

The ingress or service mesh layer is the appropriate place to introduce authentication and access control. Such security solutions are implementation-specific. Refer to the documentation from the Kubernetes security solution provider and ingress controller provider you use.