TMS GRPC API Package

Deployment Guide (1.1.0)

With every release of Triton Management Service a zip-compressed archive of the gRPC IDL files is provided. These packages can be downloaded from NVIDIA’s NGC Catalog.

It is important to use the version of the TMS gRPC API that matches the version of the TMS Server being communicated with. NVIDIA provides no guarantee of forward or backward compatibility for the programmatic interfaces of beta software. The TMS interface is still in development and expected to change.

  1. Download the gRPC API Package, using the NGC Catalog web interface or using the NGC CLI with the following command:

    ngc registry resource download-version "nvaie/triton-management-service_grpc-api-bundle:v1.1.0"

    The TMS version is specified in the last component of the command, and can be adjusted to match the version of TMS as necessary.

  2. Download the gRPC API Package, either using the web interface (provided above) or using the NGC CLI with the following command ngc registry resource download-version "nvaie/triton-management-service_grpc-api-bundle:v1.1.0". Notice that the desired version is the last component of the command, and can be adjusted to match the version of TMS as necessary.

  3. Extract the contents of the downloaded package using the commands for your OS:

    Linux

    Copy
    Copied!
                

    $ unzip ./files.zip Archive: ./files.zip inflating: files/triton-management-service_grpc-api-v1.1.0.zip $ unzip triton-management-service_grpc-api-v1.1.0.zip Archive: triton-management-service_grpc-api-v1.1.0.zip inflating: bespoke-triton.proto inflating: pooled-triton.proto inflating: lease-state.proto inflating: triton-allowlist-service.proto inflating: model.proto inflating: model-state.proto inflating: triton-pool-service.proto inflating: triton.proto inflating: lease-service.proto inflating: lease-name-service.proto inflating: common.proto inflating: lease-duration.proto inflating: error-code.proto inflating: lease-event.proto inflating: triton-state.proto

    Windows Extracting the IDL on Windows requires a tool, like 7-Zip, which can handle compressed TAR files.

    Copy
    Copied!
                

    > 7z x .\files.zip 7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20 Scanning the drive for archives: 1 file, 18579 bytes (19 KiB) Extracting archive: .\files.zip -- Path = .\files.zip Type = zip Physical Size = 18579 Everything is Ok Size: 19324 Compressed: 18579 > 7z x ./triton-management-service_grpc-api-v1.1.0.zip 7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20 Scanning the drive for archives: 1 file, 19324 bytes (19 KiB) Extracting archive: triton-management-service_grpc-api-v1.1.0.zip -- Path = triton-management-service_grpc-api-v1.1.0.zip Type = zip Physical Size = 19324 Everything is Ok Files: 15 Size: 65633 Compressed: 19324

    After extraction, validate that you have the following list of files:

    Copy
    Copied!
                

    bespoke-triton.proto lease-duration.proto lease-service.proto model.proto triton-pool-service.proto common.proto lease-event.proto lease-state.proto pooled-triton.proto triton-state.proto error-code.proto lease-name-service.proto model-state.proto triton-allowlist-service.proto triton.proto

  4. Use the protoc compiler to generate language specific code files from the provided IDL (*.proto) files. The necessary compiler and tools can be downloaded from the Protocol Buffers Release Page on GitHub. The latest, release version is v23.4. Download the version of the tools that best suits your platform.

    After all the tools are downloaded, use them to generate code in your language of choice. You can use Protocol Buffers Getting Started as a guide.

    Example for JavaScript code generation:

    Copy
    Copied!
                

    $ protoc --proto_path=proto --js_out=library=tms_model_state,binary:js_autogen proto/model-state.proto

    The command creates a JavaScript file named tms_model_state.js from proto/model-state.proto and outputs the results to a folder named js_autogen (must exist before running protoc). It assumes that the *.proto file are contained in a folder named proto, which is a child of the current working folder.

© Copyright 2023, NVIDIA. Last updated on Dec 11, 2023.