Full API

Namespaces

• Namespace nvidia
  • Namespaces

• Namespace nvidia::inferenceserver
  • Namespaces

• Namespace nvidia::inferenceserver::client
  • Classes
  • Typedefs

• Namespace nvidia::inferenceserver::custom
  • Classes

Classes and Structs

• Struct custom_initdata_struct
  • Struct Documentation

• Struct custom_payload_struct
  • Struct Documentation

• Struct InferOptions
  • Struct Documentation

• Struct InferStat
  • Struct Documentation

• Class Error
  • Class Documentation

• Class InferenceServerClient
  • Inheritance Relationships
    • Derived Types
  • Class Documentation

• Class InferenceServerGrpcClient
  • Inheritance Relationships
    • Base Type
  • Class Documentation

• Class InferenceServerHttpClient
  • Inheritance Relationships
    • Base Type
  • Class Documentation

• Class InferInput
  • Class Documentation

• Class InferRequest
  • Class Documentation

• Class InferRequestedOutput
  • Class Documentation

• Class InferResult
  • Class Documentation

• Class RequestTimers
  • Class Documentation

• Class CustomInstance
  • Class Documentation

Enums

• Enum custom_memorytype_enum
  • Enum Documentation

• Enum custom_serverparamkind_enum
  • Enum Documentation

• Enum tritonserver_batchflag_enum
  • Enum Documentation

• Enum TRITONSERVER_datatype_enum
  • Enum Documentation

• Enum TRITONSERVER_errorcode_enum
  • Enum Documentation

• Enum TRITONSERVER_loglevel_enum
  • Enum Documentation

• Enum TRITONSERVER_memorytype_enum
  • Enum Documentation

• Enum tritonserver_metricformat_enum
  • Enum Documentation

• Enum tritonserver_modelcontrolmode_enum
  • Enum Documentation

• Enum tritonserver_modelindexflag_enum
  • Enum Documentation

• Enum tritonserver_requestflag_enum
  • Enum Documentation

• Enum tritonserver_requestreleaseflag_enum
  • Enum Documentation

• Enum tritonserver_responsecompleteflag_enum
  • Enum Documentation

• Enum tritonserver_traceactivity_enum
  • Enum Documentation

• Enum tritonserver_tracelevel_enum
  • Enum Documentation

• Enum tritonserver_txn_property_flag_enum
  • Enum Documentation

Functions

• Function CustomErrorString
  • Function Documentation

• Function CustomExecute
  • Function Documentation

• Function CustomExecuteV2
  • Function Documentation

• Function CustomFinalize
  • Function Documentation

• Function CustomInitialize
  • Function Documentation

• Function CustomVersion
  • Function Documentation

• Function TRITONSERVER_DataTypeByteSize
  • Function Documentation

• Function TRITONSERVER_DataTypeString
  • Function Documentation

• Function TRITONSERVER_ErrorCode
  • Function Documentation

• Function TRITONSERVER_ErrorCodeString
  • Function Documentation

• Function TRITONSERVER_ErrorDelete
  • Function Documentation

• Function TRITONSERVER_ErrorMessage
  • Function Documentation

• Function TRITONSERVER_ErrorNew
  • Function Documentation

• Function TRITONSERVER_InferenceRequestAddInput
  • Function Documentation

• Function TRITONSERVER_InferenceRequestAddRequestedOutput
  • Function Documentation

• Function TRITONSERVER_InferenceRequestAppendInputData
  • Function Documentation

• Function TRITONSERVER_InferenceRequestCorrelationId
  • Function Documentation

• Function TRITONSERVER_InferenceRequestDelete
  • Function Documentation

• Function TRITONSERVER_InferenceRequestFlags
  • Function Documentation

• Function TRITONSERVER_InferenceRequestId
  • Function Documentation

• Function TRITONSERVER_InferenceRequestNew
  • Function Documentation

• Function TRITONSERVER_InferenceRequestPriority
  • Function Documentation

• Function TRITONSERVER_InferenceRequestRemoveAllInputData
  • Function Documentation

• Function TRITONSERVER_InferenceRequestRemoveAllInputs
  • Function Documentation

• Function TRITONSERVER_InferenceRequestRemoveAllRequestedOutputs
  • Function Documentation

• Function TRITONSERVER_InferenceRequestRemoveInput
  • Function Documentation

• Function TRITONSERVER_InferenceRequestRemoveRequestedOutput
  • Function Documentation

• Function TRITONSERVER_InferenceRequestSetCorrelationId
  • Function Documentation

• Function TRITONSERVER_InferenceRequestSetFlags
  • Function Documentation

• Function TRITONSERVER_InferenceRequestSetId
  • Function Documentation

• Function TRITONSERVER_InferenceRequestSetPriority
  • Function Documentation

• Function TRITONSERVER_InferenceRequestSetReleaseCallback
  • Function Documentation

• Function TRITONSERVER_InferenceRequestSetResponseCallback
  • Function Documentation

• Function TRITONSERVER_InferenceRequestSetTimeoutMicroseconds
  • Function Documentation

• Function TRITONSERVER_InferenceRequestTimeoutMicroseconds
  • Function Documentation

• Function TRITONSERVER_InferenceResponseDelete
  • Function Documentation

• Function TRITONSERVER_InferenceResponseError
  • Function Documentation

• Function TRITONSERVER_InferenceResponseId
  • Function Documentation

• Function TRITONSERVER_InferenceResponseModel
  • Function Documentation

• Function TRITONSERVER_InferenceResponseOutput
  • Function Documentation

• Function TRITONSERVER_InferenceResponseOutputClassificationLabel
  • Function Documentation

• Function TRITONSERVER_InferenceResponseOutputCount
  • Function Documentation

• Function TRITONSERVER_InferenceTraceActivityString
  • Function Documentation

• Function TRITONSERVER_InferenceTraceDelete
  • Function Documentation

• Function TRITONSERVER_InferenceTraceId
  • Function Documentation

• Function TRITONSERVER_InferenceTraceLevelString
  • Function Documentation

• Function TRITONSERVER_InferenceTraceModelName
  • Function Documentation

• Function TRITONSERVER_InferenceTraceModelVersion
  • Function Documentation

• Function TRITONSERVER_InferenceTraceNew
  • Function Documentation

• Function TRITONSERVER_InferenceTraceParentId
  • Function Documentation

• Function TRITONSERVER_LogIsEnabled
  • Function Documentation

• Function TRITONSERVER_LogMessage
  • Function Documentation

• Function TRITONSERVER_MemoryTypeString
  • Function Documentation

• Function TRITONSERVER_MessageDelete
  • Function Documentation

• Function TRITONSERVER_MessageSerializeToJson
  • Function Documentation

• Function TRITONSERVER_MetricsDelete
  • Function Documentation

• Function TRITONSERVER_MetricsFormatted
  • Function Documentation

• Function TRITONSERVER_ResponseAllocatorDelete
  • Function Documentation

• Function TRITONSERVER_ResponseAllocatorNew
  • Function Documentation

• Function TRITONSERVER_ServerDelete
  • Function Documentation

• Function TRITONSERVER_ServerInferAsync
  • Function Documentation

• Function TRITONSERVER_ServerIsLive
  • Function Documentation

• Function TRITONSERVER_ServerIsReady
  • Function Documentation

• Function TRITONSERVER_ServerLoadModel
  • Function Documentation

• Function TRITONSERVER_ServerMetadata
  • Function Documentation

• Function TRITONSERVER_ServerMetrics
  • Function Documentation

• Function TRITONSERVER_ServerModelBatchProperties
  • Function Documentation

• Function TRITONSERVER_ServerModelConfig
  • Function Documentation

• Function TRITONSERVER_ServerModelIndex
  • Function Documentation

• Function TRITONSERVER_ServerModelIsReady
  • Function Documentation

• Function TRITONSERVER_ServerModelMetadata
  • Function Documentation

• Function TRITONSERVER_ServerModelStatistics
  • Function Documentation

• Function TRITONSERVER_ServerModelTransactionProperties
  • Function Documentation

• Function TRITONSERVER_ServerNew
  • Function Documentation

• Function TRITONSERVER_ServerOptionsAddTensorFlowVgpuMemoryLimits
  • Function Documentation

• Function TRITONSERVER_ServerOptionsDelete
  • Function Documentation

• Function TRITONSERVER_ServerOptionsNew
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetCudaMemoryPoolByteSize
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetExitOnError
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetExitTimeout
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetGpuMetrics
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetLogError
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetLogInfo
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetLogVerbose
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetLogWarn
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetMetrics
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetMinSupportedComputeCapability
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetModelControlMode
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetModelRepositoryPath
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetPinnedMemoryPoolByteSize
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetServerId
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetStartupModel
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetStrictModelConfig
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetStrictReadiness
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetTensorFlowGpuMemoryFraction
  • Function Documentation

• Function TRITONSERVER_ServerOptionsSetTensorFlowSoftPlacement
  • Function Documentation

• Function TRITONSERVER_ServerPollModelRepository
  • Function Documentation

• Function TRITONSERVER_ServerStop
  • Function Documentation

• Function TRITONSERVER_ServerUnloadModel
  • Function Documentation

• Function TRITONSERVER_StringToDataType
  • Function Documentation

Defines

• Define CUSTOM_NO_GPU_DEVICE
  • Define Documentation

• Define CUSTOM_SERVER_PARAMETER_CNT
  • Define Documentation

• Define TRITONSERVER_EXPORT
  • Define Documentation

• Define TRTIS_CUSTOM_EXPORT
  • Define Documentation

Typedefs

• Typedef CustomErrorStringFn_t
  • Typedef Documentation

• Typedef CustomExecuteFn_t
  • Typedef Documentation

• Typedef CustomExecuteV2Fn_t
  • Typedef Documentation

• Typedef CustomFinalizeFn_t
  • Typedef Documentation

• Typedef CustomGetNextInputFn_t
  • Typedef Documentation

• Typedef CustomGetNextInputV2Fn_t
  • Typedef Documentation

• Typedef CustomGetOutputFn_t
  • Typedef Documentation

• Typedef CustomGetOutputV2Fn_t
  • Typedef Documentation

• Typedef CustomInitializeData
  • Typedef Documentation

• Typedef CustomInitializeFn_t
  • Typedef Documentation

• Typedef CustomMemoryType
  • Typedef Documentation

• Typedef CustomPayload
  • Typedef Documentation

• Typedef CustomServerParameter
  • Typedef Documentation

• Typedef CustomVersionFn_t
  • Typedef Documentation

• Typedef nvidia::inferenceserver::client::Headers
  • Typedef Documentation

• Typedef nvidia::inferenceserver::client::Parameters
  • Typedef Documentation

• Typedef TRITONSERVER_DataType
  • Typedef Documentation

• Typedef TRITONSERVER_Error_Code
  • Typedef Documentation

• Typedef TRITONSERVER_InferenceRequestReleaseFn_t
  • Typedef Documentation

• Typedef TRITONSERVER_InferenceResponseCompleteFn_t
  • Typedef Documentation

• Typedef TRITONSERVER_InferenceTraceActivity
  • Typedef Documentation

• Typedef TRITONSERVER_InferenceTraceActivityFn_t
  • Typedef Documentation

• Typedef TRITONSERVER_InferenceTraceLevel
  • Typedef Documentation

• Typedef TRITONSERVER_InferenceTraceReleaseFn_t
  • Typedef Documentation

• Typedef TRITONSERVER_LogLevel
  • Typedef Documentation

• Typedef TRITONSERVER_MemoryType
  • Typedef Documentation

• Typedef TRITONSERVER_MetricFormat
  • Typedef Documentation

• Typedef TRITONSERVER_ModelBatchFlag
  • Typedef Documentation

• Typedef TRITONSERVER_ModelControlMode
  • Typedef Documentation

• Typedef TRITONSERVER_ModelIndexFlag
  • Typedef Documentation

• Typedef TRITONSERVER_ModelTxnPropertyFlag
  • Typedef Documentation

• Typedef TRITONSERVER_RequestFlag
  • Typedef Documentation

• Typedef TRITONSERVER_RequestReleaseFlag
  • Typedef Documentation

• Typedef TRITONSERVER_ResponseAllocatorAllocFn_t
  • Typedef Documentation

• Typedef TRITONSERVER_ResponseAllocatorReleaseFn_t
  • Typedef Documentation

• Typedef TRITONSERVER_ResponseAllocatorStartFn_t
  • Typedef Documentation

• Typedef TRITONSERVER_ResponseCompleteFlag
  • Typedef Documentation