Is this page helpful?

Method 6: Container Installation#

Recommended for: Isolated development environments, reproducible builds, CI/CD pipelines, cloud deployments

Advantages:

✓ All dependencies pre-packaged ✓ Consistent environment across systems ✓ Easy to deploy and share ✓ No local installation conflicts ✓ Includes development tools and samples

Limitations:

✗ Requires Docker or compatible container runtime ✗ Requires NVIDIA Container Toolkit for GPU access ✗ Larger download size (multi-GB) ✗ Additional container overhead

Platform Support#

Supported Platforms:

  • Linux x86-64

  • Linux ARM64 (NVIDIA Jetson SBSA only; JetPack is not supported in TensorRT 11.2.1)

Prerequisites:

  • Docker (version 19.03+) or Podman installed

  • NVIDIA Container Toolkit installed and configured

  • NVIDIA GPU with appropriate drivers

Installation Steps#

Step 1: Install NVIDIA Container Toolkit (if not already installed)

Follow the platform-specific instructions in the NVIDIA Container Toolkit Installation Guide. The guide is the authoritative source and is updated as toolkit packaging and signing keys evolve; reproducing the install commands inline can quickly become outdated. Make sure to complete both the toolkit installation and the Docker runtime configuration steps before continuing.

Step 2: Pull the TensorRT NGC container

Find the latest TensorRT container on NVIDIA NGC Catalog.

docker pull nvcr.io/nvidia/tensorrt:<container-tag>

Step 3: Run the container

docker run --gpus all -it --rm \
   nvcr.io/nvidia/tensorrt:<container-tag>

This command:

  • --gpus all: Enables GPU access

  • -it: Interactive terminal

  • --rm: Removes container on exit

For production deployments that need to bound TensorRT container memory at the host level (Docker --memory, Kubernetes resources.limits.memory, cgroup memory.max), refer to Bounding TensorRT Memory in Production.

Optional: Run with specific GPU(s):

docker run --gpus '"device=0,1"' -it --rm nvcr.io/nvidia/tensorrt:<container-tag>

Verification#

For quick checks that apply to every install method, refer to Verifying Your Installation in the installation overview. The steps below are specific to NGC containers.

Inside the container, verify TensorRT installation:

Check TensorRT version:

trtexec --version

Python verification:

import tensorrt as trt
print(f"TensorRT version: {trt.__version__}")

Run a sample:

cd /workspace/tensorrt/oss
mkdir build && cd build
cmake .. -DBUILD_PARSERS=OFF -DBUILD_PLUGINS=OFF -DBUILD_SAMPLES=ON
make -j8

./sample_onnx_mnist

Troubleshooting#

For detailed information about the container, refer to the NVIDIA TensorRT Container Release Notes.

Issue: docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

  • Solution: NVIDIA Container Toolkit is not installed or not configured correctly. Restart Docker daemon after installation:

    sudo systemctl restart docker

Issue: Container cannot access GPU

  • Solution: Verify NVIDIA drivers are installed and working:

    nvidia-smi

    Ensure NVIDIA Container Toolkit runtime is set as default:

    sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Issue: Permission denied when mounting volumes

  • Solution: Add your user to the docker group:

    sudo usermod -aG docker $USER
newgrp docker