Security & Authentication

As a Developer, you are responsible for securing access to any application using the NeMo ecosystem, including an authentication layer between users and your application and securing communication between services in your application.

Rate Limiting

NeMo microservice do not impose rate limits. If you want to restrict access to your application, it is your responsibility to implement a strategy.

Ports

Text Retriever NIM uses multiple ports, but only the API Port of 8000 needs to be accessible outside of the cluster. The services’ ports are set at start up for both Text Embedding NIM and Text Reranking NIM.

Additional Security Reminders

As a Developer, you must secure your own API endpoints. We suggest using a proxy as well as HTTPS/TLS 1.2.

Incident Response

• Incident Response Plan

• Product Security

Secrets

If you deploy a Text Retriever NIM components using Helm charts, you will need at least two secrets to set up in that namespace:

• An image pull secret for NGC

• An NGC API key secret

If your cluster requires additional image pull secrets for custom init containers, you’ll need those also.

Create the secrets according to your organization’s requirements and Kubernetes secrets best practices, but for POCs and to set up secrets quickly like you can use the following, where NAMESPACE is the name of your namespace:

kubectl \--namespace NAMESPACE create secret docker-registry
registry-secret \--docker-server=nvcr.io
\--docker-username=\'\$oauthtoken\' \--docker-password=\$NGC_CLI_API_KEY

kubectl \--namespace NAMESPACE create secret generic ngc-api
\--from-literal=NGC_CLI_API_KEY=\$NGC_CLI_API_KEY

NeMo Retriever uses connection strings that may contain credentials. It is recommended to store this connection string in a secret management solution. For additional information, see NeMo Retriever documentation.

Logging

Use the following command to view the service log messages in the docker logs, where NAMESPACE is the name you gave your namespace.

docker logs NAMESPACE -f

Tip

Additional information on logging can be found in the Logging section.

See Docker security for information about securing your Docker container, including your log files.