Validation

NVIDIA AI Enterprise 2.0 or later

The following instructions are intended to be a shortcut to getting started with benchmarking. In the working directory of each benchmark, there is a README file (named either README.md or README.txt) that provides more details of data download, pre-processing, and running the code.

For demonstration purposes, we will run Deep Learning inferencing. Please refer to the NVIDIA Multi-node Training Deployment <deployment-guide-multi-node:multi-node-overview> document for additional information regarding running Deep Learning training workflows.

  • The container used in this example nvcr.io/nvaie/tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>.

  • Binary needed is included with the container at /workspace/tensorrt/bin.

  • The Resnet50 model prototxt and caffemodel files are within the container at /workspace/tensorrt/data/resnet50.

  • The command may take several minutes to run because NVIDIA® TensorRT™ is building the optimized plan before running. If you wish to see what it is doing, add --verbose to the command.

Commands to the Run Test

Copy
Copied!
            

$ sudo podman pull nvcr.io/nvaie/tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> $ sudo podman run --gpus all -it --rm -v $(pwd):/work nvcr.io/nvaie/tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> # cd /workspace/tensorrt/data/resnet50 (to exit container, type “exit”) # /workspace/tensorrt/bin/trtexec --batch=128 --iterations=400 --workspace=1024 --percentile=99 --deploy=ResNet50_N2.prototxt --model=ResNet50_fp32.caffemodel --output=prob --int8

Interpreting the Results

Results are reported in time to infer the given batch size. To convert to images per second, compute BATCH_SIZE/AVERAGE_TIME. The Average Time can be found as the mean GPU Compute value of the tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> inferencing output.

© Copyright 2022-2023, NVIDIA. Last updated on Sep 11, 2023.