This page is for Self-Hosted, version v3.0 (default).
For other versions, use one of these documentation indexes:
- v3.0 (default): https://docs.nvidia.com/run-ai/self-hosted/v3.0/llms.txt
- v2.26: https://docs.nvidia.com/run-ai/self-hosted/v2.26/llms.txt
- v2.25: https://docs.nvidia.com/run-ai/self-hosted/v2.25/llms.txt
- v2.24: https://docs.nvidia.com/run-ai/self-hosted/v2.24/llms.txt
Inference workloads deploy trained models into a production environment to generate predictions from live data. These workloads are prioritized over Trainings and Workspaces during scheduling. NVIDIA Run:ai Inference workloads support auto-scaling to maintain service-level agreements (SLAs) by dynamically adjusting resources as demand changes.
