Examples

We provide step-by-step examples that demonstrate how to use various features of Model Navigator. For the sake of readability and accessibility, we use a simple torch.nn.Linear model as an example. These examples illustrate how to optimize, test and deploy the model on the PyTriton and Triton Inference Server.

Step-by-step examples

1. Optimize model

2. Optimize model and verify model

3. Optimize model and save package

4. Load and optimize package

5. Optimize and serve model on PyTriton

6. Optimize and serve model on Triton Inference Server

7. Optimize model and use for offline inference

8. Optimize PyTorch QAT model

9. Custom configuration for optimize

10. Inplace Optimize of single model

11. Inplace Optimize of models pipeline

Example models

Inside the example/models directory you can find ready to use example models in various frameworks.

Python:

• Identity Model

PyTorch:

• BART (Inplace Optimize)

• BERT

• Linear Model

• ResNet50

• ResNet50 (Inplace Optimize)

• Stable Diffusion (Inplace Optimize)

• Whisper (Inplace Optimize)

TensorFlow:

• Linear Model

• EfficientNet

• BERT

JAX:

• Linear Model

• GPT-2

ONNX:

• Identity Model