# TAO Toolkit Launcher

TAO Toolkit encapsulates DNN training pipelines that may be developed across different training platforms. In order to abstract the details from TAO Toolkit users, TAO Toolkit now is packaged with a launcher CLI. The CLI is a python3 wheel package that may be installed using the python-pip. When installed, the launcher CLI abstracts the user from having to instantiate and run several TAO containers and map the commands accordingly.

In this release of TAO Toolkit, the TAO package includes two underlying dockers, based on the training framework. Each docker contains entrypoints to a task, which runs the sub-tasks associated with them. The tasks and sub-tasks follow the following cascaded structure

Copy
Copied!

tao <task> <sub-task> <args>


The tasks are broadly divided into computer vision and conversational AI. For example, DetectNet_v2 is a computer vision task for object detection in TAO Toolkit, which supports subtasks such as train, prune, evaluate, export etc. When the user executes a command, for example tao detectnet_v2 train --help, the TAO Toolkit Launcher does the following:

1. Pulls the required TAO container with the entrypoint for DetectNet_v2

2. Creates an instance of the container

3. Runs the detectnet_v2 entrypoint with the train sub-task

You may visualize the user interaction diagram for the TAO Launcher when running training and evaluation for a DetectNet_v2 model as follows:

Similarly, the interaction diagram for training a QuartzNet speech_to_text model is as follows:

The following sections cover supported commands and configuring the launcher.

## Running the launcher

Once the launcher has been installed, the workflow to run the launcher is as follows.

1. Listing the tasks supported in the docker.

After installing the launcher, you will now be able to list the tasks that are supported in the TAO Toolkit Launcher. The output of tao --help command is as follows:

Copy
Copied!

usage: tao [-h]
{list,stop,info,augment,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet,
retinanet,speech_to_text,ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny}
...

Launcher for TAO

optional arguments:
-h, --help            show this help message and exit

{list,stop,info,augment,classification,detectnet_v2,dssd,emotionnet,faster_rcnn,fpenet,gazenet,gesturenet,heartratenet
ssd,text_classification,converter,token_classification,unet,yolo_v3,yolo_v4,yolo_v4_tiny}


2. Configuring the launcher instance.

Running Deep Neural Networks implies working on large datasets. These datasets are usually present network share drives, with significantly higher storage capacity. Since the TAO Toolkit Launcher users docker containers under the hood, these drives/mount points need to be mapped to the docker. The launcher instance can be configured in the ~/.tao_mounts.json file.

The launcher config file consists of 3 sections, namely:

• Mounts

• Envs

• DockerOptions

The Mounts parameter defines the paths in the local machine, that should be mapped to the docker. This is a list of json dictionaries containing the source path in the local machine and the destination path that is mapped for the TAO Toolkit commands.

The Envs parameter defines the environement variables to be set to the respective TAO Toolkit docker. This is also a list of dictionaries. Each dictionary entry has 2 key-value pairs defined.

• variable: The name of the environment variable you would like to set

• value: The value of the environment variable

The DockerOptions parameter defines the options to be set when invoking the docker instance. This parameter is a dictionary containing key-value pair of the parameter and option to set. Currently, the TAO Toolkit Launcher only allows users to configure the following parameters.

• shm_size: Defines the shared memory size of the docker. If this parameter isn’t set, then the TAO Toolkit instance allocates 64MB by default. We recommend setting this as "16G", thereby allocating 16GB of shared memory.

• ulimits: Defines the user limits in the docker. This parameter corresponds to the ulimit parameters in /etc/security/limits.conf. We recommend users set memlock to -1 and stack to 67108864.

• user: Defines the user id and group id of the user to run the commands in the docker. By default, if this parameter isn’t defined in the ~/.tao_mounts.json the uid and gid of the root user. However, this would mean that when directories created by the TAO dockers would be set to root permissions. If you would like to set the user in the docker to be the same as the host user, please set this parameter as “UID:GID”, where UID and GID can be obtained from the command line by running id -u and id -g.

• ports: This parameter defines the ports in the docker to be mounted to the host.

You may specify this parameter as a dictionary containig the map between the port in the docker to the port in the host machine. For example, if you wish to expose port 8888 and port 8000, this parameter would look as follows:

Copy
Copied!

"ports":{
"8888":"8888",
"8000":"8000"
}


Please use the following code block as a sample for the ~/.tao_mounts.json file. In this mounts sample, we define 3 drives, an environment variable called CUDA_DEVICE_ORDER. For DockerOptions we set shared memory size of 16G, user limits and set the host user’s permission. We also bind the port 8888 from the docker to the host.

Copy
Copied!

{
"Mounts": [
{
"source": "/path/to/your/data",
"destination": "/workspace/tao-experiments/data"
},
{
"source": "/path/to/your/local/results",
"destination": "/workspace/tao-experiments/results"
},
{
"source": "/path/to/config/files",
"destination": "/workspace/tao-experiments/specs"
}
],
"Envs": [
{
"variable": "CUDA_DEVICE_ORDER",
"value": "PCI_BUS_ID"
}
],
"DockerOptions": {
"shm_size": "16G",
"ulimits": {
"memlock": -1,
"stack": 67108864
},
"user": "1000:1000",
"ports": {
"8888": 8888
}
}
}


Similarly, a sample config file containing 2 mount points and no docker options is as below.

Copy
Copied!

{
"Mounts": [
{
"source": "/path/to/your/experiments",
"destination": "/workspace/tao-experiments"
},
{
"source": "/path/to/config/files",
"destination": "/workspace/tao-experiments/specs"
}
]
}


Once you have installed the TAO Toolkit Launcher, you may now run the tasks supported by TAO Toolkit using the following command format.

Copy
Copied!

tao <task> <subtask> <cli_args>


To view the sub-tasks supported by a certain task, you may run the command following the template

For example: Listing the tasks of detectnet_v2, the outputs is as follows:

Copy
Copied!