TMS GRPC API Package

With every release of Triton Management Service a zip-compressed archive of the gRPC IDL files is provided. These packages can be downloaded from NVIDIA’s NGC Catalog.

It is important to use the version of the TMS gRPC API that matches the version of the TMS Server being communicated with. NVIDIA provides no guarantee of forward or backward compatibility for the programmatic interfaces of beta software. The TMS interface is still in development and expected to change.

1. Download the gRPC API Package, using the NGC Catalog web interface or using the NGC CLI with the following command:

   ngc registry resource download-version "nvaie/triton-management-service_grpc-api-bundle:v1.1.0"

   The TMS version is specified in the last component of the command, and can be adjusted to match the version of TMS as necessary.

2. Download the gRPC API Package, either using the web interface (provided above) or using the NGC CLI with the following command ngc registry resource download-version "nvaie/triton-management-service_grpc-api-bundle:v1.1.0". Notice that the desired version is the last component of the command, and can be adjusted to match the version of TMS as necessary.

3. Extract the contents of the downloaded package using the commands for your OS:

   Linux

   Copy

   Copied!

               
   
               
   $ unzip ./files.zip
   Archive:  ./files.zip
   inflating: files/triton-management-service_grpc-api-v1.1.0.zip
   $ unzip triton-management-service_grpc-api-v1.1.0.zip
   Archive:  triton-management-service_grpc-api-v1.1.0.zip
   inflating: bespoke-triton.proto
   inflating: pooled-triton.proto
   inflating: lease-state.proto
   inflating: triton-allowlist-service.proto
   inflating: model.proto
   inflating: model-state.proto
   inflating: triton-pool-service.proto
   inflating: triton.proto
   inflating: lease-service.proto
   inflating: lease-name-service.proto
   inflating: common.proto
   inflating: lease-duration.proto
   inflating: error-code.proto
   inflating: lease-event.proto
   inflating: triton-state.proto
           

   Windows Extracting the IDL on Windows requires a tool, like 7-Zip, which can handle compressed TAR files.

   Copy

   Copied!

               
   
               
   > 7z x .\files.zip
   
   7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20
   
   Scanning the drive for archives:
   1 file, 18579 bytes (19 KiB)
   
   Extracting archive: .\files.zip
   --
   Path = .\files.zip
   Type = zip
   Physical Size = 18579
   
   Everything is Ok
   
   Size:       19324
   Compressed: 18579
   
   > 7z x ./triton-management-service_grpc-api-v1.1.0.zip
   
   7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20
   
   Scanning the drive for archives:
   1 file, 19324 bytes (19 KiB)
   
   Extracting archive: triton-management-service_grpc-api-v1.1.0.zip
   --
   Path = triton-management-service_grpc-api-v1.1.0.zip
   Type = zip
   Physical Size = 19324
   
   Everything is Ok
   
   Files: 15
   Size:       65633
   Compressed: 19324
           

   After extraction, validate that you have the following list of files:

   Copy

   Copied!

               
   
               
   bespoke-triton.proto
   lease-duration.proto
   lease-service.proto
   model.proto
   triton-pool-service.proto
   common.proto
   lease-event.proto
   lease-state.proto
   pooled-triton.proto
   triton-state.proto
   error-code.proto
   lease-name-service.proto
   model-state.proto
   triton-allowlist-service.proto
   triton.proto
           

4. Use the protoc compiler to generate language specific code files from the provided IDL (*.proto) files. The necessary compiler and tools can be downloaded from the Protocol Buffers Release Page on GitHub. The latest, release version is v23.4. Download the version of the tools that best suits your platform.

   After all the tools are downloaded, use them to generate code in your language of choice. You can use Protocol Buffers Getting Started as a guide.

   Example for JavaScript code generation:

   Copy

   Copied!

               
   
               
   $ protoc --proto_path=proto --js_out=library=tms_model_state,binary:js_autogen proto/model-state.proto
           

   The command creates a JavaScript file named tms_model_state.js from proto/model-state.proto and outputs the results to a folder named js_autogen (must exist before running protoc). It assumes that the *.proto file are contained in a folder named proto, which is a child of the current working folder.