BYOM Converter

Bring Your Own Model (BYOM) is a Python-based package that converts any open-source ONNX model to a TAO-comaptible model. The TAO BYOM Converter provides a CLI to import an ONNX model and convert it to Keras. The converted model is stored in .tltb format, which is based on EFF.

Currently, the following models are supported by the TAO BYOM Converter:

• Classification

• UNet

More tasks supported by the TAO, such as Object Detection, will be added in future releases.

Software Requirements

Installing through pip

The TAO BYOM Converter is a Python pip package that is hosted on the official PyPI. It is a Python 3 only package, capable of running on python 3.6.

1. Create a new conda environment using miniconda.

   1. Follow the instructions in this link to set up a conda environment using a miniconda.

   2. Once you have installed miniconda, create a new environment by setting the Python version to 3.6.

      Copy

      Copied!

                  
      
                  
      conda create -n launcher python=3.6
              

   3. Activate the conda environment that you have just created.

      Copy

      Copied!

                  
      
                  
      conda activate launcher
              

   4. Once you have activated your conda environment, the command prompt should show the name of your conda environment.

      Copy

      Copied!

                  
      
                  
      (launcher) py-3.6.9 desktop:
              

   5. When you are done with you session, you may deactivate your conda environment using the deactivate command:

      Copy

      Copied!

                  
      
                  
      conda deactivate
              

   6. You may re-instantiate this created conda environment using the following command.

   Copy

   Copied!

               
   
               
   conda activate launcher
           

   
   

2. Install the Tensorflow 1.15.x package from NVIDIA, which is hosted on nvidia-pyindex.

   Copy

   Copied!

               
   
               
   pip3 install nvidia-pyindex
   pip3 install nvidia-tensorflow
           

3. Install the TAO BYOM Converter Python package, called nvidia-tao-byom.

   Copy

   Copied!

               
   
               
   pip3 install nvidia-tao-byom
           

   Note

   If you had installed an older version of the nvidia-tao-byom launcher, you may upgrade to the latest version by running the following command.

   Copy

   Copied!

               
   
               
   pip3 install --upgrade nvidia-tao-byom
           

4. Invoke the entrypoints using the tao_byom command.

   Copy

   Copied!

               
   
               
   tao_byom --help
           

   
   

   The sample output of the above command is as follows:

   Copy

   Copied!

               
   
               
   usage: BYOM Converter [-h] -m ONNX_MODEL_FILE -n MODEL_NAME -r RESULTS_DIR
                 [--verbose] [-c CUSTOM_META] [-k KEY]
                 [-p PENULTIMATE_NODE]
                 [-ki {glorot_uniform,glorot_normal,he_uniform,he_normal, \
                 zeros,random_uniform,random_normal,constant,ones,identity}]
                 [-fn [FREEZE_NODE [FREEZE_NODE ...]]] [-fb]
   
   Convert onnx model into TAO Model.
   
   optional arguments:
   -h, --help            show this help message and exit
   -m ONNX_MODEL_FILE, --onnx_model_file ONNX_MODEL_FILE
                           Onnx model path to the pre-trained weights.
   -n MODEL_NAME, --model_name MODEL_NAME
                           Name of the architecture inside onnx file
   -r RESULTS_DIR, --results_dir RESULTS_DIR
                           Path to a folder where converted Keras model will be
                           stored.
   --verbose             log level NOTSET
   -c CUSTOM_META, --custom_meta CUSTOM_META
                           Path to custom meta json file that contains info about
                           custom layer implementation
   -k KEY, --key KEY     Key to encrpyt tltb file
   -p PENULTIMATE_NODE, --penultimate_node PENULTIMATE_NODE
                           Name of ONNX node corresponding to the penultimate
                           layer
   -ki {glorot_uniform,glorot_normal,he_uniform,he_normal,zeros,random_uniform,\
                           random_normal,constant,ones,identity}, \
                           --kernel_initializer {glorot_uniform,glorot_normal, he_uniform,\
                           he_normal,zeros,random_uniform,random_normal,constant,ones,identity}
                           Type of kernel initializer used to initialize Conv,
                           ConvTranspose, and Gemm
   -fn [FREEZE_NODE [FREEZE_NODE ...]], --freeze_node [FREEZE_NODE [FREEZE_NODE ...]]
                           List of ONNX nodes to freeze. Examples: -i item1 item2
   -fb, --freeze_bn      Whether to freeze every BatchNorm in the model.
           

   
   

            

            
export PATH=$PATH:~/.local/bin
        

Software Requirements

Installing via pip

The TAO BYOM Converter is a Python3 pip package that is hosted on the official PyPI repository. It is capable of running on python 3.8 for TF2.

1. Create a new conda environment using miniconda.

   1. Follow the instructions in this link to set up a conda environment using a miniconda.

   2. Once you have installed miniconda, create a new environment by setting the Python version to 3.8.

      Copy

      Copied!

                  
      
                  
      conda create -n launcher python=3.8
              

   3. Activate the conda environment that you just created.

      Copy

      Copied!

                  
      
                  
      conda activate launcher
              

   4. Once you have activated your conda environment, the command prompt should show the name of the environment.

      Copy

      Copied!

                  
      
                  
      (launcher) py-3.8.9 desktop:
              

   5. When you are done with you session, you may deactivate your conda environment using the deactivate command:

      Copy

      Copied!

                  
      
                  
      conda deactivate
              

   6. You may re-instantiate this created conda environment using the following command.

   Copy

   Copied!

               
   
               
   conda activate launcher
           

   
   

2. Install the Tensorflow 2.9.x package and necessary CUDA related dependencies.

   Copy

   Copied!

               
   
               
   conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0
   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/
   pip3 install tensorflow==2.9.1
           

3. Install the TAO BYOM Converter Python package, named nvidia-tao-byom.

   Copy

   Copied!

               
   
               
   pip3 install nvidia-tao-byom
           

   Note

   If you have an older version of the nvidia-tao-byom launcher installed, you may upgrade to the latest version by running the following command.

   Copy

   Copied!

               
   
               
   pip3 install --upgrade nvidia-tao-byom
           

4. Invoke the entrypoints using the tao_byom command.

   Copy

   Copied!

               
   
               
   tao_byom --help
           

   
   

   The sample output of the above command is as follows:

   Copy

   Copied!

               
   
               
   usage: BYOM Converter [-h] -m ONNX_MODEL_FILE -n MODEL_NAME -r RESULTS_DIR
                 [--verbose] [-c CUSTOM_META] [-k KEY]
                 [-p PENULTIMATE_NODE]
                 [-ki {glorot_uniform,glorot_normal,he_uniform,he_normal, \
                 zeros,random_uniform,random_normal,constant,ones,identity}]
                 [-fn [FREEZE_NODE [FREEZE_NODE ...]]] [-fb]
   
   Convert onnx model into TAO Model.
   
   optional arguments:
   -h, --help            show this help message and exit
   -m ONNX_MODEL_FILE, --onnx_model_file ONNX_MODEL_FILE
                           Onnx model path to the pre-trained weights.
   -n MODEL_NAME, --model_name MODEL_NAME
                           Name of the architecture inside onnx file
   -r RESULTS_DIR, --results_dir RESULTS_DIR
                           Path to a folder where converted Keras model will be
                           stored.
   --verbose             log level NOTSET
   -c CUSTOM_META, --custom_meta CUSTOM_META
                           Path to custom meta json file that contains info about
                           custom layer implementation
   -k KEY, --key KEY     Key to encrpyt tltb file
   -p PENULTIMATE_NODE, --penultimate_node PENULTIMATE_NODE
                           Name of ONNX node corresponding to the penultimate
                           layer
   -ki {glorot_uniform,glorot_normal,he_uniform,he_normal,zeros,random_uniform,\
                           random_normal,constant,ones,identity}, \
                           --kernel_initializer {glorot_uniform,glorot_normal, he_uniform,\
                           he_normal,zeros,random_uniform,random_normal,constant,ones,identity}
                           Type of kernel initializer used to initialize Conv,
                           ConvTranspose, and Gemm
   -fn [FREEZE_NODE [FREEZE_NODE ...]], --freeze_node [FREEZE_NODE [FREEZE_NODE ...]]
                           List of ONNX nodes to freeze. Examples: -i item1 item2
   -fb, --freeze_bn      Whether to freeze every BatchNorm in the model.
           

   
   

            

            
export PATH=$PATH:~/.local/bin
        

You will need an ONNX model to run the TAO BYOM Converter. There are multiple ways to get an ONNX model for model architectures you want. You may first look through models supported in the official ONNX model zoo. ONNX provides various vision model architectures that were exported from different Deep Learning frameworks.

In addition to the official ONNX model website, you can also export your model from the framework of your choice to ONNX. For PyTorch, refer to their API for exporting to ONNX. For Tensorflow, refer to ONNX supported library called tensorflow-onnx.

Use the tao_byom command to convert an open-source ONNX model to TAO compatible model:

            

            
tao_byom [-h] -m <onnx model path>
              -k <encoding key>
              -r <result directory>
              -n <name of the model>
              [-p <name of penultimate ONNX node>]
              [-c <custom meta json>]
              [-ki <type of kernel initialization>]
              [-fn <list of ONNX nodes to freeze>]
              [-fb]
              [--verbose]
        

Required Arguments

• -m, --onnx_model_file: The path to an ONNX model file

• -k, --key: A user-specific encoding key to save a .tltb model

• -r, --results_dir: The path to a folder where the converted outputs should be written

• -n, --model_name: The name of the model architecture

Optional Arguments

• --penultimate_node, -p: The name of ONNX node corresponding to the penultimate. If provided, the converter will interecept the output to the provided node. Provide this argument if you want to finetune the open source model on a different dataset through the TAO. The definition of --penultimate_node can vary depending on the task. For classification, it is often the last activation before the global average pooling. TAO will add a layer at the end to update the size of the classification head depending on the number of classes in the target dataset. For UNet, it is the last convolution before the last softmax/sigmoid (if there’s an activation before the output).

• --custom_meta, -c: The path to a custom meta JSON file that contains information about custom layer implementation. Provide this argument if there’s an ONNX node that the BYOM Converter doesn’t currently support. You can bring your own implementations to extend the functionality of BYOM Converter. For more details, refer to the BYOM Example repo.

• --kernel_initializer, -ki: The type of kernel initializer used to initialize Conv, ConvTranspose, and Gemm. Available options are glorot_uniform, glorot_normal, he_uniform, he_normal, zeros, random_uniform, random_normal, constant, ones, identity. For more information about kernel initialization, refer to the Keras documentation.

• --freeze_node, -fn: A list of ONNX nodes to freeze. ONNX nodes are limited to trainable nodes (e.g. Conv, ConvTranspose, and Gemm). The weights of layers corresponding to the provided ONNX nodes will not be updated during training. This argument is similar to freeze_blocks in the classification model_config.

• --freeze_bn, -fb: A flag specifying whether to freeze every Batch Normalization in the model. This argument is similar to freeze_bn in the classification model_config.

• --verbose: Prints detailed output. The log level is set to NOTSET.

• -h, --help: Prints the help message.

            

            
INFO: Difference between the original ONNX and converted Keras model is larger than the set threshold 0.0001 with error of 0.000356.
INFO: This may be due to difference in deep learning frameworks. If the error is not far from the threshold, you may proceed with
training with TAO. If difference is large, please post an issue on the forum and link your original model.
        

A sample workflow for converting open-source models through TAO BYOM Converter is described in the TAO BYOM Example repository.

The below table shows all supported ONNX operators in TAO BYOM. This may be updated in future releases. If you wish to add a new ONNX operator to TAO BYOM, refer to the example in the Bring your Own Layer section of the TAO BYOM documentation.