Security for NeMo Microservices

This page provides security guidelines and best practices for deploying and managing NeMo microservices in production environments.

Security Considerations

• The NeMo microservices do not impose rate limits. You must implement a rate-limiting strategy to restrict access to your application.

• The NeMo microservices do not have an internal notion of a user. To restrict authorization to specific endpoints or users, implement an external mechanism such as an Envoy proxy.

• The NeMo Data Store microservice does not provide object-class-specific access controls. All items reside within a single access control boundary.

• The NeMo microservices, by design, can access all content in the NeMo Data Store microservice, including LoRA adapters, training data, evaluation data, and evaluation results. This access is required for model evaluation. Carefully weigh the risk of data exposure from serving customized models directly to production against the overhead of a separate deployment.

• The NeMo microservices are not intended to be internet-facing. Deploy them as the logic (middle) tier in a three-tier architecture.

• You are responsible for securing access to any application using the microservices. This includes:

  • Implementing an authentication layer between users and your application

  • Applying required authorization controls

  • Securing communication between services in your application

Note

Refer to the NVIDIA Product Security page for information about subscribing to bulletins and updates, managing vulnerabilities, and reporting vulnerabilities.

Default Network Ports

The following table lists the default network ports for each microservice or default database. You can override these port numbers during deployment.

Network Port	Microservice
443/TCP	NeMo Admission Service API
2746/TCP	Argo Workflows Server
3000/TCP	NeMo Data Store API
7331/TCP	NeMo Evaluator API
7331/TCP	NeMo Guardrails API
8000/TCP	NIM for LLMs API
8000/TCP	NeMo Retriever Text Embedding API
8000/TCP	NeMo Retriever Text Reranking API
8000/TCP	NeMo Customizer API
8000/TCP	NeMo Entity Store API
8443/TCP	NeMo Operator metrics
8080/TCP	Volcano Scheduler metrics
9009/TCP	NeMo Customizer callback

By default, the NeMo Microservices Helm Chart configures databases with the following network ports. Alternatively, you can configure each microservice to use an external database during installation.

Network Port	Database
5432/TCP	NeMo Customizer Database
5432/TCP	NeMo Data Store Database
5432/TCP	NeMo Evaluator Database
5432/TCP	NeMo Entity Store Database
9091/TCP	Milvus metrics
19530/TCP	Milvus API

By default, the NeMo Microservices Helm Chart installs an open telemetry collector, which uses the following network ports:

• 4317/TCP

• 4318/TCP

• 6831/UDP

• 9411/TCP

• 14250/TCP

• 14268/TCP

Refer to the OpenTelemetry Documentation for more information.