Is this page helpful?

Security and Authentication for NVIDIA NIM for Nemotron Parse#

Use this documentation to learn about security and authentication for NVIDIA NIM for Nemotron Parse.

As a developer, you are responsible for securing access to any application using the NeMo ecosystem, including an authentication layer between users and your application and securing communication between services in your application.

Important

As a Developer, you must secure your own API endpoints. We suggest using a proxy as well as HTTPS/TLS 1.2.

Rate Limiting#

NIMs do not impose rate limits. If you want to restrict access to your application, implement rate limiting in a gateway or proxy. To limit the number of requests waiting for batch processing, set the batcher queue depth with NIM_SERVER_MAX_QUEUE_SIZE.

Ports#

Set the HTTP listen address with NIM_SERVER_BIND_ADDR in host:port format. The default is 0.0.0.0:8000. Expose this address only to clients that require access to the HTTP API.

To enable the optional KServe V2 gRPC API, set NIM_SERVER_GRPC_BIND_ADDR in host:port format. The gRPC API is disabled when this variable is unset. Expose the gRPC address only to clients that require access to that API.

Incident Response#

Secrets#

If you deploy Nemotron Parse by using Helm charts, you need an image pull secret for NGC in that namespace. You need a model-download secret only when your model download provider uses a credential. Set up the following secrets:

  • An image pull secret for NGC. This secret is always required, because the container image comes from NGC.

  • One model-download secret, depending on the provider:

    • NGC: An NGC API key secret.

    • Hugging Face with a token: A Hugging Face token secret.

    • Anonymous Hugging Face: No model-download secret. Set nim.ngcAPISecret to an empty string.

If your cluster requires additional image pull secrets for custom init containers, you’ll need those also.

Create the secrets according to your organization’s requirements and Kubernetes secrets best practices, but for POCs and to set up secrets quickly you can use the following commands, where NAMESPACE is the name of your namespace. Create the ngc-api secret only for NGC downloads, and the hf-api secret only for authenticated Hugging Face downloads.

kubectl \
    create secret -n NAMESPACE docker-registry ngc-secret  \
    --docker-server=nvcr.io --docker-username='$oauthtoken' \
    --docker-password=$NGC_API_KEY

kubectl \
    create secret -n NAMESPACE generic ngc-api  \
    --from-literal=NGC_API_KEY=$NGC_API_KEY

kubectl \
    create secret -n NAMESPACE generic hf-api  \
    --from-literal=HF_TOKEN=$HF_TOKEN

For Hugging Face deployment examples, refer to Deploying with Hugging Face.

We recommend that you store credentials in a secret management solution.

Logging#

Use the following command to view the service log messages using Docker.

docker logs <container-id> -f

Alternatively, use the following command to view the service log messages for a container deployed on Kubernetes.

kubectl logs -n <namespace> <pod-name> -f

Tip

Additional information on logging can be found in the Logging section.

See Docker security for information about securing your Docker container, including your log files.