Troubleshooting Guide¶

To share feedback or ask questions about this release, access our NVIDIA TLT Developer Forum.

NGC¶

Before pulling assets from NGC, ensure you run the following commands and follow the required prompts
ngc config set docker login nvcr.io
When running ngc config set, the NGC CLI may not prompt the user to configure the team and org. In this case, users may run into an error when downloading models saying
Missing org - If apikey is set, org is also required.
Please maintain a back-up of your existing NGC API key from the ngc config at ~/.ngc/config and clear the ngc config by running the following command.
ngc config clear

TLT Launcher¶

The launcher CLI abstracts the user’s interaction with the container and brings out the entrypoints inside the respective dockers.

When installing the TLT launcher, make sure to install the nvidia-pyindex before installing nvidia-tlt. The TLT launcher package is not hosted on PyPI. Otherwise you may run into this error.

Collecting nvidia-tlt
Downloading nvidia-tlt-0.0.1.dev4.tar.gz (4.0 kB)
Complete output (15 lines):
Traceback (most recent call last):
..
..
RuntimeError:
###########################################################################################
The package you are trying to install is only a placeholder project on PyPI.org repository.
This package is hosted on NVIDIA Python Package Index.

This package can be installed as:
```
$ pip install nvidia-pyindex
$ pip install nvidia-tlt
```
###########################################################################################
----------------------------------------

Make sure to set your python to python3 when running the launcher. The TLT launcher is strictly a python3 package
When installing the tlt launcher to your host machine’s native python3 as opposed to the recommended route of using virtual environment, you may get an error saying that tlt binary wasn’t found. This is because the path to your tlt binary installed by pip wasn’t added to the PATH environment variable in your local machine. In this case, please run the following command:
export PATH=$PATH:/home/$USER/.local/bin
Make sure to have all the paths required by the TLT docker to be exposed to it via the ~/.tlt_mounts.json. The launcher by default, does not have any paths mapped to it.
When running the TLT launcher, for CV applications, we recommend setting the TLT launcher to run as the user’s host account so that the user may have permissions to edit the results directories and collaterals generated by the TLT dockers. The dockers by default are instantiated as root, so the users will need sudo access to edit the results path etc. For more information on configuring the user, please refer to the Configuring the launcher section.

When running the any TLT command for the first time, the launcher pulls the container from the docker registry. This process can take a few minutes. The log will look as follows

2021-02-24 08:16:04,270 [INFO] tlt.components.docker_handler.docker_handler: The required docker doesn't exist locally/the manifest has changed. Pulling a new docker.
2021-02-24 08:16:04,270 [INFO] tlt.components.docker_handler.docker_handler: Pulling the required container. This may take several minutes if you're doing this for the first time. Please wait here.