Optimize Torch linear model and save package

In this example, we show how to save a .nav package. It contains all the information about the performed optimizations and the models themselves. The Navigator package is portable and transferable, and can be reused locally or transferred to another machine.

Only base models and those with minimal latency and maximal throughput are saved.

Base formats are:

• Torch:

  • TorchScript

  • ONNX

• TensorFlow2:

  • SavedModel

• JAX:

  • SavedModel

• ONNX:

  • ONNX

We recommend running this example in NVIDIA NGC PyTorch container. To run the example, simply run the optimize.py script:

./optimize.py --output-path linear.nav