NVIDIA TensorRT Inference Server
1.2.0 -0000000
Version select:
  • Documentation home

User Guide

  • Quickstart
    • Prerequisites
    • Using A Prebuilt Docker Container
    • Building From Source Code
    • Run TensorRT Inference Server
    • Verify Inference Server Is Running Correctly
    • Getting The Client Examples
    • Running The Image Classification Example
  • Installing the Server
    • Installing Prebuilt Containers
  • Running the Server
    • Example Model Repository
    • Running The Inference Server
    • Running The Inference Server On A System Without A GPU
    • Checking Inference Server Status
  • Client Libraries and Examples
    • Getting the Client Libraries and Examples
      • Build Using Dockerfile
      • Build Using Makefile
      • Download From GitHub
    • Image Classification Example Application
    • Ensemble Image Classification Example Application
    • Performance Example Application
    • Client API
      • String Datatype
      • Client API for Stateful Models
  • Models And Schedulers
    • Stateless Models
    • Stateful Models
    • Ensemble Models
  • Model Repository
    • Modifying the Model Repository
    • Model Versions
    • Framework Model Definition
      • TensorRT Models
      • TensorFlow Models
      • Caffe2 Models
      • TensorRT/TensorFlow Models
      • ONNX Models
    • Custom Backends
      • Custom Backend API
      • Example Custom Backend
    • Ensemble Backends
  • Model Configuration
    • Generated Model Configuration
    • Datatypes
    • Reshape
    • Version Policy
    • Instance Groups
    • Scheduling And Batching
      • Default Scheduler
      • Dynamic Batcher
      • Sequence Batcher
      • Ensemble Scheduler
    • Optimization Policy
  • Inference Server API
    • Health
    • Status
    • Inference
    • Stream Inference
  • Metrics

Developer Guide

  • Architecture
    • Concurrent Model Execution
  • Custom Operations
    • TensorRT
    • TensorFlow
  • Building
    • Building the Server
      • Incremental Builds
    • Building the Client Libraries and Examples
      • Build Using Dockerfile
      • Build Using Makefile
    • Building the Documentation
  • Testing
    • Generate QA Model Repositories
    • Build QA Container
    • Run QA Container
  • Contributing
    • Coding Convention

Reference

  • Capabilities
  • Protobuf API
    • HTTP/GRPC API
    • Model Configuration
    • Status
  • C++ API
    • Class Hierarchy
    • File Hierarchy
    • Full API
      • Namespaces
        • Namespace nvidia
        • Namespace nvidia::inferenceserver
        • Namespace nvidia::inferenceserver::client
      • Classes and Structs
        • Struct custom_initdata_struct
        • Struct custom_payload_struct
        • Struct Result::ClassResult
        • Struct InferContext::Stat
        • Class Error
        • Class InferContext
        • Class InferContext::Input
        • Class InferContext::Options
        • Class InferContext::Output
        • Class InferContext::Request
        • Class InferContext::Result
        • Class InferGrpcContext
        • Class InferGrpcStreamContext
        • Class InferHttpContext
        • Class ProfileContext
        • Class ProfileGrpcContext
        • Class ProfileHttpContext
        • Class ServerHealthContext
        • Class ServerHealthGrpcContext
        • Class ServerHealthHttpContext
        • Class ServerStatusContext
        • Class ServerStatusGrpcContext
        • Class ServerStatusHttpContext
      • Enums
        • Enum custom_serverparamkind_enum
      • Functions
        • Function CustomErrorString
        • Function CustomExecute
        • Function CustomFinalize
        • Function CustomInitialize
        • Function nvidia::inferenceserver::client::operator<<
      • Defines
        • Define CUSTOM_NO_GPU_DEVICE
        • Define CUSTOM_SERVER_PARAMETER_CNT
      • Typedefs
        • Typedef CustomErrorStringFn_t
        • Typedef CustomExecuteFn_t
        • Typedef CustomFinalizeFn_t
        • Typedef CustomGetNextInputFn_t
        • Typedef CustomGetOutputFn_t
        • Typedef CustomInitializeData
        • Typedef CustomInitializeFn_t
        • Typedef CustomPayload
        • Typedef CustomServerParameter
        • Typedef nvidia::inferenceserver::CorrelationID
        • Typedef nvidia::inferenceserver::DimsList
      • Directories
        • Directory src
        • Directory clients
        • Directory c++
        • Directory servables
        • Directory custom
      • Files
        • File custom.h
        • File request.h
        • File request_grpc.h
        • File request_http.h
  • Python API
    • Client
NVIDIA TensorRT Inference Server
  • Docs »
  • Search

© Copyright 2018, NVIDIA Corporation

Built with Sphinx using a theme provided by Read the Docs.