> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/distributed-inferences/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/run-ai/_mcp/server.

# Distributed Inferences

Distributed inference enables running inference workloads across multiple pods, typically to scale model serving beyond a single container or node. This approach is useful when a single instance cannot meet resource requirements.NVIDIA Run:ai supports this model using Leader Worker Set (LWS). 
Each pod plays a specific role, either as a leader or worker, and together they form a coordinated service. NVIDIA Run:ai manages the orchestration and configuration of these pods to ensure efficient and scalable inference execution


## API Docs

- Distributed Inferences [Create a distributed inference.](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/distributed-inferences/create-distributed-inference.md)
- Distributed Inferences [Get a distributed inference data.](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/distributed-inferences/get-distributed-inference.md)
- Distributed Inferences [Delete a distributed inference.](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/distributed-inferences/delete-distributed-inference.md)
- Distributed Inferences [Update distributed inference spec.](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/distributed-inferences/update-distributed-inference-spec.md)

## OpenAPI Specification

The raw OpenAPI 3.1 specification for this API is available at:
- [OpenAPI JSON](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/distributed-inferences/openapi.json)
- [OpenAPI YAML](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/distributed-inferences/openapi.yaml)