Security for NeMo Platform | NVIDIA NeMo Platform

This page provides security guidelines and best practices for deploying and managing NeMo Platform in production environments.

Security Considerations

The NeMo Platform does not impose rate limits. You must implement a rate-limiting strategy to restrict access to your application.
The NeMo Platform does not have an internal notion of a user. To restrict authorization to specific endpoints or users, implement an external mechanism such as an Envoy proxy.
The NeMo Data Store microservice does not provide object-class-specific access controls. All items reside within a single access control boundary.
The NeMo Platform, by design, can access all content in the NeMo Data Store microservice, including LoRA adapters, training data, evaluation data, and evaluation results. This access is required for model evaluation. Carefully weigh the risk of data exposure from serving customized models directly to production against the overhead of a separate deployment.
The NeMo Platform is not intended to be internet-facing. Deploy them as the logic (middle) tier in a three-tier architecture.
You are responsible for securing access to any application using the microservices. This includes:
Implementing an authentication layer between users and your application
Applying required authorization controls
Securing communication between services in your application

Refer to the NVIDIA Product Security page for information about subscribing to bulletins and updates, managing vulnerabilities, and reporting vulnerabilities.

Default Network Ports

The following table lists the default network ports for each microservice or default database. You can override these port numbers during deployment.

Network Port	Microservice
443/TCP	NeMo Admission Service API
3000/TCP	NeMo Data Store API
7331/TCP	NeMo Evaluator API
7331/TCP	NeMo Guardrails API
8000/TCP	NIM API
8000/TCP	NeMo Retriever Text Embedding API
8000/TCP	NeMo Retriever Text Reranking API
8000/TCP	NeMo Customizer API
8000/TCP	NeMo Entity Store API
8443/TCP	NeMo Operator metrics
8080/TCP	Volcano Scheduler metrics
9009/TCP	NeMo Customizer callback

By default, the NeMo Platform Helm Chart configures databases with the following network ports. Alternatively, you can configure each microservice to use an external database during installation.

Network Port	Database
5432/TCP	NeMo Customizer Database
5432/TCP	NeMo Data Store Database
5432/TCP	NeMo Evaluator Database
5432/TCP	NeMo Entity Store Database
9091/TCP	Milvus metrics
19530/TCP	Milvus API

By default, the NeMo Platform Helm Chart installs an open telemetry collector, which uses the following network ports:

4317/TCP
4318/TCP
6831/UDP
9411/TCP
14250/TCP
14268/TCP

Refer to the OpenTelemetry Documentation for more information.