Is this page helpful?

Security and Authentication for NVIDIA NIM for Object Detection#

Use this documentation to learn about security and authentication for NVIDIA NIM for Object Detection.

As a developer, you are responsible for securing access to any application using the NeMo ecosystem, including an authentication layer between users and your application and securing communication between services in your application.

Important

As a Developer, you must secure your own API endpoints. We suggest using a proxy as well as HTTPS/TLS 1.2.

Rate Limiting#

Object Detection NIM does not impose user-level rate limits. Use your application gateway or proxy to implement rate limits. To limit the NIM batcher queue depth, configure NIM_SERVER_MAX_QUEUE_SIZE.

Ports#

The HTTP API listens on NIM_SERVER_BIND_ADDR, which defaults to 0.0.0.0:8000. If KServe V2 gRPC is enabled, it listens on NIM_SERVER_GRPC_BIND_ADDR.

Expose only the ports required by your deployment.

Incident Response#

Secrets#

Store credentials in a secret management solution. The default auto provider setting selects Hugging Face when HF_TOKEN is available, or NGC when only NGC_API_KEY is available. To force a provider, use NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=hf with HF_TOKEN or NIM_ENGINE_MODEL_DOWNLOAD_PROVIDER=ngc with NGC_API_KEY.

For Kubernetes deployments, store credentials in Kubernetes secrets and mount them as environment variables.

Logging#

Use the following command to view the service log messages using Docker.

docker logs <container-id> -f

Alternatively, use the following command to view the service log messages for a container deployed on Kubernetes.

kubectl logs -n <namespace> <pod-name> -f

Tip

Additional information on logging can be found in the Logging section.

See Docker security for information about securing your Docker container, including your log files.