C++ API¶

Class Hierarchy¶

- Namespace nvidia
  - Namespace nvidia::inferenceserver
    - Namespace nvidia::inferenceserver::client
      - Struct InferOptions
      - Struct InferStat
      - Class Error
      - Class InferenceServerClient
      - Class InferenceServerGrpcClient
      - Class InferenceServerHttpClient
      - Class InferInput
      - Class InferRequest
      - Class InferRequestedOutput
      - Class InferResult
      - Class RequestTimers
    - Namespace nvidia::inferenceserver::custom
      - Class CustomInstance
- Struct cudaIpcMemHandle_t
- Struct custom_initdata_struct
- Struct custom_payload_struct
- Enum custom_memorytype_enum
- Enum custom_serverparamkind_enum
- Enum TRITONSERVER_datatype_enum
- Enum TRITONSERVER_errorcode_enum
- Enum TRITONSERVER_memorytype_enum
- Enum tritonserver_metricformat_enum
- Enum tritonserver_modelcontrolmode_enum
- Enum tritonserver_modelindexflag_enum
- Enum tritonserver_requestflag_enum
- Enum tritonserver_traceactivity_enum
- Enum tritonserver_tracelevel_enum

File Hierarchy¶

- Directory src
  - Directory backends
    - Directory custom
      - File custom.h
  - Directory clients
    - Directory c++
      - Directory library
        File common.h
        File grpc_client.h
        File http_client.h
  - Directory core
    - File tritonserver.h
  - Directory custom
    - Directory sdk
      - File custom_instance.h

Full API¶

Namespaces¶

Namespace nvidia
- Namespaces

Namespace nvidia::inferenceserver
- Namespaces

Namespace nvidia::inferenceserver::client
- Classes
- Typedefs

Namespace nvidia::inferenceserver::custom
- Classes

Classes and Structs¶

Struct cudaIpcMemHandle_t
- Struct Documentation

Struct custom_initdata_struct
- Struct Documentation

Struct custom_payload_struct
- Struct Documentation

Struct InferOptions
- Struct Documentation

Struct InferStat
- Struct Documentation

Class Error
- Class Documentation

Class InferenceServerClient
- Inheritance Relationships
  - Derived Types
- Class Documentation

Class InferenceServerGrpcClient
- Inheritance Relationships
  - Base Type
- Class Documentation

Class InferenceServerHttpClient
- Inheritance Relationships
  - Base Type
- Class Documentation

Class InferInput
- Class Documentation

Class InferRequest
- Class Documentation

Class InferRequestedOutput
- Class Documentation

Class InferResult
- Class Documentation

Class RequestTimers
- Class Documentation

Class CustomInstance
- Class Documentation

Functions¶

Function CustomErrorString
- Function Documentation

Function CustomExecute
- Function Documentation

Function CustomExecuteV2
- Function Documentation

Function CustomFinalize
- Function Documentation

Function CustomInitialize
- Function Documentation

Function CustomVersion
- Function Documentation

Function TRITONSERVER_DataTypeByteSize
- Function Documentation

Function TRITONSERVER_DataTypeString
- Function Documentation

Function TRITONSERVER_ErrorCode
- Function Documentation

Function TRITONSERVER_ErrorCodeString
- Function Documentation

Function TRITONSERVER_ErrorDelete
- Function Documentation

Function TRITONSERVER_ErrorMessage
- Function Documentation

Function TRITONSERVER_ErrorNew
- Function Documentation

Function TRITONSERVER_InferenceRequestAddInput
- Function Documentation

Function TRITONSERVER_InferenceRequestAddRequestedOutput
- Function Documentation

Function TRITONSERVER_InferenceRequestAppendInputData
- Function Documentation

Function TRITONSERVER_InferenceRequestCorrelationId
- Function Documentation

Function TRITONSERVER_InferenceRequestDelete
- Function Documentation

Function TRITONSERVER_InferenceRequestFlags
- Function Documentation

Function TRITONSERVER_InferenceRequestId
- Function Documentation

Function TRITONSERVER_InferenceRequestNew
- Function Documentation

Function TRITONSERVER_InferenceRequestPriority
- Function Documentation

Function TRITONSERVER_InferenceRequestRemoveAllInputData
- Function Documentation

Function TRITONSERVER_InferenceRequestRemoveAllInputs
- Function Documentation

Function TRITONSERVER_InferenceRequestRemoveAllRequestedOutputs
- Function Documentation

Function TRITONSERVER_InferenceRequestRemoveInput
- Function Documentation

Function TRITONSERVER_InferenceRequestRemoveRequestedOutput
- Function Documentation

Function TRITONSERVER_InferenceRequestSetCorrelationId
- Function Documentation

Function TRITONSERVER_InferenceRequestSetFlags
- Function Documentation

Function TRITONSERVER_InferenceRequestSetId
- Function Documentation

Function TRITONSERVER_InferenceRequestSetPriority
- Function Documentation

Function TRITONSERVER_InferenceRequestSetReleaseCallback
- Function Documentation

Function TRITONSERVER_InferenceRequestSetResponseCallback
- Function Documentation

Function TRITONSERVER_InferenceRequestSetTimeoutMicroseconds
- Function Documentation

Function TRITONSERVER_InferenceRequestTimeoutMicroseconds
- Function Documentation

Function TRITONSERVER_InferenceResponseDelete
- Function Documentation

Function TRITONSERVER_InferenceResponseError
- Function Documentation

Function TRITONSERVER_InferenceResponseId
- Function Documentation

Function TRITONSERVER_InferenceResponseModel
- Function Documentation

Function TRITONSERVER_InferenceResponseOutput
- Function Documentation

Function TRITONSERVER_InferenceResponseOutputClassificationLabel
- Function Documentation

Function TRITONSERVER_InferenceResponseOutputCount
- Function Documentation

Function TRITONSERVER_InferenceTraceActivityString
- Function Documentation

Function TRITONSERVER_InferenceTraceDelete
- Function Documentation

Function TRITONSERVER_InferenceTraceId
- Function Documentation

Function TRITONSERVER_InferenceTraceLevelString
- Function Documentation

Function TRITONSERVER_InferenceTraceModelName
- Function Documentation

Function TRITONSERVER_InferenceTraceModelVersion
- Function Documentation

Function TRITONSERVER_InferenceTraceNew
- Function Documentation

Function TRITONSERVER_InferenceTraceParentId
- Function Documentation

Function TRITONSERVER_MemoryTypeString
- Function Documentation

Function TRITONSERVER_MessageDelete
- Function Documentation

Function TRITONSERVER_MessageSerializeToJson
- Function Documentation

Function TRITONSERVER_MetricsDelete
- Function Documentation

Function TRITONSERVER_MetricsFormatted
- Function Documentation

Function TRITONSERVER_ResponseAllocatorDelete
- Function Documentation

Function TRITONSERVER_ResponseAllocatorNew
- Function Documentation

Function TRITONSERVER_ServerDelete
- Function Documentation

Function TRITONSERVER_ServerInferAsync
- Function Documentation

Function TRITONSERVER_ServerIsLive
- Function Documentation

Function TRITONSERVER_ServerIsReady
- Function Documentation

Function TRITONSERVER_ServerLoadModel
- Function Documentation

Function TRITONSERVER_ServerMetadata
- Function Documentation

Function TRITONSERVER_ServerMetrics
- Function Documentation

Function TRITONSERVER_ServerModelConfig
- Function Documentation

Function TRITONSERVER_ServerModelIndex
- Function Documentation

Function TRITONSERVER_ServerModelIsReady
- Function Documentation

Function TRITONSERVER_ServerModelMetadata
- Function Documentation

Function TRITONSERVER_ServerModelStatistics
- Function Documentation

Function TRITONSERVER_ServerNew
- Function Documentation

Function TRITONSERVER_ServerOptionsAddTensorFlowVgpuMemoryLimits
- Function Documentation

Function TRITONSERVER_ServerOptionsDelete
- Function Documentation

Function TRITONSERVER_ServerOptionsNew
- Function Documentation

Function TRITONSERVER_ServerOptionsSetCudaMemoryPoolByteSize
- Function Documentation

Function TRITONSERVER_ServerOptionsSetExitOnError
- Function Documentation

Function TRITONSERVER_ServerOptionsSetExitTimeout
- Function Documentation

Function TRITONSERVER_ServerOptionsSetGpuMetrics
- Function Documentation

Function TRITONSERVER_ServerOptionsSetLogError
- Function Documentation

Function TRITONSERVER_ServerOptionsSetLogInfo
- Function Documentation

Function TRITONSERVER_ServerOptionsSetLogVerbose
- Function Documentation

Function TRITONSERVER_ServerOptionsSetLogWarn
- Function Documentation

Function TRITONSERVER_ServerOptionsSetMetrics
- Function Documentation

Function TRITONSERVER_ServerOptionsSetMinSupportedComputeCapability
- Function Documentation

Function TRITONSERVER_ServerOptionsSetModelControlMode
- Function Documentation

Function TRITONSERVER_ServerOptionsSetModelRepositoryPath
- Function Documentation

Function TRITONSERVER_ServerOptionsSetPinnedMemoryPoolByteSize
- Function Documentation

Function TRITONSERVER_ServerOptionsSetServerId
- Function Documentation

Function TRITONSERVER_ServerOptionsSetStartupModel
- Function Documentation

Function TRITONSERVER_ServerOptionsSetStrictModelConfig
- Function Documentation

Function TRITONSERVER_ServerOptionsSetStrictReadiness
- Function Documentation

Function TRITONSERVER_ServerOptionsSetTensorFlowGpuMemoryFraction
- Function Documentation

Function TRITONSERVER_ServerOptionsSetTensorFlowSoftPlacement
- Function Documentation

Function TRITONSERVER_ServerPollModelRepository
- Function Documentation

Function TRITONSERVER_ServerStop
- Function Documentation

Function TRITONSERVER_ServerUnloadModel
- Function Documentation

Function TRITONSERVER_StringToDataType
- Function Documentation

Defines¶

Define CUSTOM_NO_GPU_DEVICE
- Define Documentation

Define CUSTOM_SERVER_PARAMETER_CNT
- Define Documentation

Define TRITONSERVER_EXPORT
- Define Documentation

Define TRTIS_CUSTOM_EXPORT
- Define Documentation

Typedefs¶

Typedef CustomErrorStringFn_t
- Typedef Documentation

Typedef CustomExecuteFn_t
- Typedef Documentation

Typedef CustomExecuteV2Fn_t
- Typedef Documentation

Typedef CustomFinalizeFn_t
- Typedef Documentation

Typedef CustomGetNextInputFn_t
- Typedef Documentation

Typedef CustomGetNextInputV2Fn_t
- Typedef Documentation

Typedef CustomGetOutputFn_t
- Typedef Documentation

Typedef CustomGetOutputV2Fn_t
- Typedef Documentation

Typedef CustomInitializeData
- Typedef Documentation

Typedef CustomInitializeFn_t
- Typedef Documentation

Typedef CustomMemoryType
- Typedef Documentation

Typedef CustomPayload
- Typedef Documentation

Typedef CustomServerParameter
- Typedef Documentation

Typedef CustomVersionFn_t
- Typedef Documentation

Typedef nvidia::inferenceserver::client::Headers
- Typedef Documentation

Typedef nvidia::inferenceserver::client::Parameters
- Typedef Documentation

Typedef TRITONSERVER_DataType
- Typedef Documentation

Typedef TRITONSERVER_Error_Code
- Typedef Documentation

Typedef TRITONSERVER_InferenceRequestReleaseFn_t
- Typedef Documentation

Typedef TRITONSERVER_InferenceResponseCompleteFn_t
- Typedef Documentation

Typedef TRITONSERVER_InferenceTraceActivity
- Typedef Documentation

Typedef TRITONSERVER_InferenceTraceActivityFn_t
- Typedef Documentation

Typedef TRITONSERVER_InferenceTraceLevel
- Typedef Documentation

Typedef TRITONSERVER_InferenceTraceReleaseFn_t
- Typedef Documentation

Typedef TRITONSERVER_MemoryType
- Typedef Documentation

Typedef TRITONSERVER_MetricFormat
- Typedef Documentation

Typedef TRITONSERVER_ModelControlMode
- Typedef Documentation

Typedef TRITONSERVER_ModelIndexFlag
- Typedef Documentation

Typedef TRITONSERVER_RequestFlag
- Typedef Documentation

Typedef TRITONSERVER_ResponseAllocatorAllocFn_t
- Typedef Documentation

Typedef TRITONSERVER_ResponseAllocatorReleaseFn_t
- Typedef Documentation