Triton Image Allowlist

The Triton image allowlist is a management feature that controls which Triton images can be used to create Triton pools and bespoke Triton instances. This feature gives administrators some basic controls over which images can and cannot be used when creating Triton instances. If an image is not in the allowlist, it cannot be used to create a new Triton instance. The Triton allowlist service is used to inspect and modify the allowlist.

The list of operations for the Triton Allowlist Service includes the following:

• TritonAllowlist/Append adds a Triton container image to the list of the allowed Triton container images.

  The server returns success or failure depending on whether the requested image could be added to the allowlist.

  Attempting to add images that are already present in the allowlist does not result in any changes.

• TritonAllowlist/List returns a list of the allowed Triton container images.

  This RPC begins streaming a response after the request has been received.

  Each response message contains a Triton image that belongs to the list.

• TritonAllowlist/Remove deletes a Triton container image from the list of the allowed Triton container images.

  The server returns success or failure depending on whether the image could be removed from the allowlist.

Initially, the allowlist only contains the default Triton image, as configured during installation. This can be seen by running tmsctl allowlist list. For example, to verify your installation you can run the command to return the default image:

            

            
$ tmsctl allowlist list
nvcr.io/nvidia/tritonserver:23.09-py3
        

At this stage, new Triton instances can only be created with nvcr.io/nvidia/tritonserver:23.09-py3 as the Triton image. For example, trying to use nvcr.io/nvidia/tritonserver:23.08-py3 fails.

            

            
$ tmsctl lease create --triton-image nvcr.io/nvidia/tritonserver:23.08-py3 -m "name=$MODEL_NAME,uri=$MODEL_URI"
fatal: Requested Triton container image ("nvcr.io/nvidia/tritonserver:23.08-py3") is unreachable or not provided in a supported format. Unreachable container images either do not exist or require privileges not granted to the server. (triton_options_bespoke.triton.container_image @ Acquire)
        

New entries can be added using tmsctl allowlist add. For example:

            

            
$ tmsctl allowlist add nvcr.io/nvidia/tritonserver:23.08-py3
Added nvcr.io/nvidia/tritonserver:23.08-py3

$ tmsctl allowlist list
nvcr.io/nvidia/tritonserver:23.08-py3
nvcr.io/nvidia/tritonserver:23.09-py3
        

After adding images, you can create new bespoke Triton instances and Triton pools specifying nvcr.io/nvidia/tritonserver:23.08-py3 as the image. For example:

            

            
$ tmsctl lease create --triton-image nvcr.io/nvidia/tritonserver:23.08-py3 -m "name=$MODEL_NAME,uri=$MODEL_URI"
Lease da21b2c0e68b49ffa8f0f6db0b030128
State: Valid
Expires: 2023-10-18T15:43:53Z
Triton: triton-6d8c9d13.tmsns.svc.cluster.local
<nvcr.io/nvidia/tritonserver:23.08-py3>

Models:
Name          Url                                 Status
<model_name>  <model_url>                         Ready
        

You can remove entries using tmsctl allowlist rm. For example:

            

            
$ tmsctl allowlist rm nvcr.io/nvidia/tritonserver:23.10-py3
Removed nvcr.io/nvidia/tritonserver:23.10-py3

$ tmsctl allowlist list
nvcr.io/nvidia/tritonserver:23.09-py3
        

After removing images, any attempts to create new bespoke Tritons or Triton pools specifying the removed image causes a failure.