tao/tao-toolkit/text/excerpts/tensorrt_oss_on_x86.html

Building TensorRT OSS on x86:

1. Install Cmake (>=3.13).

   Note

   TensorRT OSS requires cmake >= v3.13, so install cmake 3.13 if your cmake version is lower than 3.13c

   sudo apt remove --purge --auto-remove cmake
   wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
   tar xvf cmake-3.13.5.tar.gz
   cd cmake-3.13.5/
   ./configure
   make -j$(nproc)
   sudo make install
   sudo ln -s /usr/local/bin/cmake /usr/bin/cmake
   

2. Get GPU architecture. The GPU_ARCHS value can be retrieved by the deviceQuery CUDA sample:

   cd /usr/local/cuda/samples/1_Utilities/deviceQuery
   sudo make
   ./deviceQuery
   

   If the /usr/local/cuda/samples doesn’t exist in your system, you could download deviceQuery.cpp from this GitHub repo. Compile and run deviceQuery.

   nvcc deviceQuery.cpp -o deviceQuery
   ./deviceQuery
   

   This command will output something like this, which indicates the GPU_ARCHS is 75 based on CUDA Capability major/minor version.

   Detected 2 CUDA Capable device(s)
   
   Device 0: "Tesla T4"
     CUDA Driver Version / Runtime Version          10.2 / 10.2
     CUDA Capability Major/Minor version number:    7.5
   

3. Build TensorRT OSS:

   git clone -b 21.08 https://github.com/nvidia/TensorRT
   cd TensorRT/
   git submodule update --init --recursive
   export TRT_SOURCE=`pwd`
   cd $TRT_SOURCE
   mkdir -p build && cd build
   

   Note

   Make sure your GPU_ARCHS from step 2 is in TensorRT OSS CMakeLists.txt. If GPU_ARCHS is not in TensorRT OSS CMakeLists.txt, add -DGPU_ARCHS=<VER> as below, where <VER> represents GPU_ARCHS from step 2.

   /usr/local/bin/cmake .. -DGPU_ARCHS=xy  -DTRT_LIB_DIR=/usr/lib/x86_64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=`pwd`/out
   make nvinfer_plugin -j$(nproc)
   

   After building ends successfully, libnvinfer_plugin.so* will be generated under \`pwd\`/out/.

4. Replace the original libnvinfer_plugin.so*:

   sudo mv /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.8.x.y ${HOME}/libnvinfer_plugin.so.8.x.y.bak   // backup original libnvinfer_plugin.so.x.y
   sudo cp $TRT_SOURCE/`pwd`/out/libnvinfer_plugin.so.8.m.n  /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.8.x.y
   sudo ldconfig