Validation

NVIDIA AI Enterprise 2.0 or later

The following instructions are intended to be a shortcut to getting started with benchmarking. In the working directory of each benchmark, there is a README file (named either README.md or README.txt) that provides more details of data download, pre-processing, and running the code.

For demonstration purposes, we will run Deep Learning inferencing. Please refer to the NVIDIA Multi-node Training Deployment <deployment-guide-multi-node:multi-node-overview> document for additional information regarding running Deep Learning training workflows.

• The container used in this example nvcr.io/nvaie/tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>.

• Binary needed is included with the container at /workspace/tensorrt/bin.

• The Resnet50 model prototxt and caffemodel files are within the container at /workspace/tensorrt/data/resnet50.

• The command may take several minutes to run because NVIDIA® TensorRT™ is building the optimized plan before running. If you wish to see what it is doing, add --verbose to the command.

Commands to the Run Test

            

            
$ sudo podman pull nvcr.io/nvaie/tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>
$ sudo podman run --gpus all -it --rm -v $(pwd):/work nvcr.io/nvaie/tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>
# cd /workspace/tensorrt/data/resnet50 (to exit container, type “exit”)
# /workspace/tensorrt/bin/trtexec --batch=128 --iterations=400 --workspace=1024 --percentile=99 --deploy=ResNet50_N2.prototxt --model=ResNet50_fp32.caffemodel --output=prob --int8
        

Interpreting the Results

Results are reported in time to infer the given batch size. To convert to images per second, compute BATCH_SIZE/AVERAGE_TIME. The Average Time can be found as the mean GPU Compute value of the tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> inferencing output.