NVIDIA Triton Inference Server

Release Notes

The actual inference server is packaged in the Triton Inference Server container. This document provides information about how to set up and run the Triton inference server container, from the prerequisites to running the container. The release notes also provide a list of key features, packaged software in the container, software enhancements and improvements, known issues, and how to run the Triton Inference Server 2.43.0 (V2 API) for the 24.02 and earlier releases. The Triton inference server container is released monthly to provide you with the latest NVIDIA deep learning software libraries and GitHub code contributions that have been sent upstream. The libraries and contributions have all been tested, tuned, and optimized.

Inference Server

Documentation - Latest Release

This Triton Inference Server documentation focuses on the Triton inference server and its benefits. The inference server is included within the inference server container. This guide provides step-by-step instructions for pulling and running the Triton inference server container, along with the details of the model store and the inference API.

Documentation – Pre-release

This is the GitHub pre-release documentation for Triton inference server. This documentation is an unstable documentation preview for developers and is updated continuously to be in sync with the Triton inference server in GitHub.

Licenses

SLA

This document is the Software License Agreement (SLA) for NVIDIA Triton Inference Server. The following contains specific license terms and conditions for NVIDIA Triton Inference Server. By accepting this agreement, you agree to comply with all the terms and conditions applicable to the specific product(s) included herein.

BSD License

This document is the Berkeley Software Distribution (BSD) license for NVIDIA Triton Inference Server. The following contains specific license terms and conditions for NVIDIA Triton Inference Server open sourced. By accepting this agreement, you agree to comply with all the terms and conditions applicable to the specific product(s) included herein.

Archives

Documentation Archives

This Archives document provides access to previously released Triton inference server documentation versions.