Red Hat Enterprise Linux with KVM Deployment Guide (0.1.0)

NVIDIA AI Enterprise 2.0 or later

The following instructions are intended to be a shortcut to getting started with benchmarking. In the working directory of each benchmark, there is a README file (named either or README.txt) that provides more details of data download, pre-processing, and running the code.

For demonstration purposes, we will run Deep Learning inferencing. Please refer to the NVIDIA Multi-node Training Deployment <deployment-guide-multi-node:multi-node-overview> document for additional information regarding running Deep Learning training workflows.

  • The container used in this example<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>.

  • Binary needed is included with the container at /workspace/tensorrt/bin.

  • The Resnet50 model prototxt and caffemodel files are within the container at /workspace/tensorrt/data/resnet50.

  • The command may take several minutes to run because NVIDIA® TensorRT™ is building the optimized plan before running. If you wish to see what it is doing, add --verbose to the command.

Commands to the Run Test


$ sudo podman pull<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> $ sudo podman run --gpus all -it --rm -v $(pwd):/work<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> # cd /workspace/tensorrt/data/resnet50 (to exit container, type “exit”) # /workspace/tensorrt/bin/trtexec --batch=128 --iterations=400 --workspace=1024 --percentile=99 --deploy=ResNet50_N2.prototxt --model=ResNet50_fp32.caffemodel --output=prob --int8

Interpreting the Results

Results are reported in time to infer the given batch size. To convert to images per second, compute BATCH_SIZE/AVERAGE_TIME. The Average Time can be found as the mean GPU Compute value of the tensorrt-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> inferencing output.

Previous Advance Framework Configuration
Next Support and Services
© Copyright 2024, NVIDIA. Last updated on Apr 2, 2024.