CoreTypes.h File Reference

Fundamental NvNeural data types are declared here. More...

#include <cstddef>
#include <cstdint>
#include <tuple>
#include <type_traits>
#include <vector>
#include <initializer_list>

Go to the source code of this file.

Classes
struct	nvneural::ActivationDesc
	Structure describing an activation function (IActivationFunction) for tool interfaces and network builders. More...
struct	nvneural::BaseParamTypeDesc
	For implementers, this struct needs to be replicated in each xParamTypeDesc right now. More...
struct	nvneural::DimensionParamTypeDesc
	The DimensionParamTypeDesc further describes a parameter for the plugin and is pointed to from the higher level parameter descriptor ParamDesc. More...
struct	nvneural::EnumParamTypeDesc
	The EnumParamTypeDesc further describes a parameter for the plugin and is pointed to from the higher level parameter descriptor ParamDesc. More...
class	nvneural::Float16
	Float16 is a CPU-side implementation of half-precision floating point (aka FP16). More...
struct	nvneural::FusingRule
	Defines a fusing rule that can be applied during model loading. More...
class	nvneural::IActivationFunction
	IActivationFunction represents a functor that can perform a specific activation function. More...
class	nvneural::IApiVersionQuery
	The IApiVersionQuery interface provides this object's compiled-in ApiVersion. More...
class	nvneural::IBackendDeviceIdentifier
	IBackendDeviceIdentifier is an opaque identifier for an active GPU. More...
class	nvneural::IClassRegistry
	Common class registry for instantiating objects and collecting plugin-provided descriptors. More...
class	nvneural::IImage
	IImage represents a bitmap image in system memory. More...
class	nvneural::ILayerList
	ILayerList represents an immutable collection of ILayer pointers. More...
class	nvneural::ILibraryContext
	ILibraryContext is a generic library handle wrapper. More...
class	nvneural::ILogger
	Logger interface class. More...
class	nvneural::INetworkBackend
	INetworkBackend is a runtime-specific interface for CUDA, DirectX, or other system- specific operations needed during inference. More...
class	nvneural::INetworkBackend2
	INetworkBackend2 is a revision of INetworkBackend. More...
class	nvneural::INetworkDebuggerList
	INetworkDebuggerList represents an immutable collection of INetworkDebugger pointers. More...
class	nvneural::INetworkRuntime
	INetworkRuntime is a subset of the basic network interface that is accessible from layer classes during network evaluation. More...
struct	nvneural::InterfaceOf< TInterface >
	Helper template to aid retrieval of interface IDs. More...
class	nvneural::IParameterNode
	Represents a serialized parameter block in a model definition. More...
class	nvneural::IRefObject
	Base class for all objects, similar to COM's IUnknown. More...
class	nvneural::IRuntimeOptionsHost
	IRuntimeOptionsHost is an interface that provides runtime option communication from the layer to the host. More...
class	nvneural::IStringList
	IStringList represents an immutable collection of strings. More...
class	nvneural::IWeightsData
	IWeightsData represents a binary buffer provided by IWeightsLoader. More...
class	nvneural::IWeightsLoader
	IWeightsLoader is an interface that provides weights to layers. More...
struct	nvneural::LayerDesc
	This structure describes an ILayer for tools and network builders. More...
struct	nvneural::LayerInputDesc
	Describes a single input terminal for a layer. LayerDesc objects contain arrays of this structure. More...
struct	nvneural::MemoryTrackingData
	Structure describing details of an object's memory allocation behavior. More...
struct	nvneural::ParamDesc
	ParamDesc describes a parameter for the plugin, giving it a name, default value, type, and, if needed, a pointer to greater detail. More...
struct	nvneural::PrototypeDesc
	This structure represent a prototype layer version (backend, tensor format and kernel type), it should be passed to ClassRegistry using registerPrototype. More...
struct	nvneural::StringParamTypeDesc
	The StringParamTypeDesc further describes a parameter for the plugin and is pointed to from the higher level parameter descriptor ParamDesc. More...
struct	nvneural::TensorDimension
	TensorDimension describes the dimensions of a four-dimensional image tensor. More...
struct	nvneural::TensorFormat
	TensorFormat describes a specific tensor shape (element type and layout). More...

Namespaces
	nvneural::common_categories
	The categories below are pre-defined and used when defining a CategoryList.

Typedefs
using	nvneural::MemoryHandle = MemoryHandle__type *
	Opaque typedef used to represent INetworkBackend memory handles.
typedef bool(*	nvneural::PrototypeRuntimeValidatorFunction) (const INetwork *pNetwork)
	This function type represents a runtime validator. More...

Enumerations
enum class	nvneural::ActivationFunctionId : std::uint32_t {   None = 0 , ReLU = 1 , LeakyReLU = 2 , LeakySigmoid = 3 ,   LeakyTanh = 4 , Tanh = 5 , Sigmoid = 6 , Softmax = 7 ,   Sine = 8 , CustomBase = 0x4000u }
	Enumeration describing common activation functions. More...
enum class	nvneural::CompilationLevel : std::uint16_t { CompiledBinary = 0 , JitIntermediate = 1 , JitSource = 2 }
	This enum represent kernel type contained by a prototype layer. More...
enum class	nvneural::ImageSpace : std::uint32_t { Invalid = 0 , RgbNormalized = 1 , RgbCentered = 2 , CustomBase = 0x4000u }
	ImageSpace values refer to specific tensor representations of image data. More...
enum class	nvneural::LayerInputType : std::uint16_t { Primary = 0 , PrimaryInfinite = 1 , Secondary = 2 , CustomBase = 0x4000u }
	This enum describes what type a particular input is for the layer. More...
enum class	nvneural::MemorySemantic : std::uint32_t {   Unspecified = 0u , Tensor = 1u , Weights = 2u , Ephemeral = 3u ,   CustomBase = 0x4000u }
	Describes the intended purpose of allocated GPU memory. More...
enum class	nvneural::NetworkBackendId : std::uint32_t {   Invalid = 0 , TestOnly = 1 , Cpu = 2 , Cuda = 3 ,   CustomBase = 0x4000u }
	Enumeration describing common network backends. More...
enum class	nvneural::NeuralResult : std::int32_t { Success = 0 , Failure = 1 , Unsupported = 2 }
	NeuralResult is a generic success/failure result type similar to COM HRESULT. More...
enum class	nvneural::ParamType : std::uint16_t {   String = 0 , SizeT = 1 , Float = 2 , Enumeration = 3 ,   Dimension = 4 , Bool = 5 , MaxParam , CustomBase = 0x4000u }
	This enum describes what type a particular parameter. More...
enum class	nvneural::TensorDataLayout : std::uint16_t {   Invalid = 0 , TestOnly = 1 , Nchw = 2 , Nhwc = 3 ,   CustomBase = 0x4000u }
	Enumeration describing common tensor data layouts. More...
enum class	nvneural::TensorDataType : std::uint16_t {   Invalid = 0 , TestOnly = 1 , Float = 2 , Half = 3 ,   CompressedPng = 7 , UncompressedRgb = 8 , CustomBase = 0x4000u }
	Enumeration describing common tensor element types. More...

Functions
constexpr bool	nvneural::failed (NeuralResult result) noexcept
	Helper function akin to COM's FAILED() macro.
constexpr bool	nvneural::succeeded (NeuralResult result) noexcept
	Helper function akin to COM's SUCCEEDED() macro.

Variables
const char *const	nvneural::common_categories::Analysis = "Analysis"
	Category for visualization and analysis layers.
const std::uint32_t	nvneural::ApiVersion = 1338
	API version set. More...
const char *const	nvneural::common_categories::DataImageProcessing = "Data/Image Processing"
	Category for layers that do image processing.
const char *const	nvneural::common_categories::DefaultHidden = "Hidden"
	Category for layers that should be hidden in the editor by default.
const char *const	nvneural::common_categories::DevelopmentAids = "Development Aids"
	Category for layers that aid framework development.
const char *const	nvneural::common_categories::DomainSpecific = "Domain Specific"
	Category for layers that implement specialized operations.
const char *const	nvneural::common_categories::Examples = "Examples"
	Category for demonstration layers.
const char *const	nvneural::common_categories::FormatConversion = "Format Conversion"
	Category for layers that permute and shuffle data.
const char *const	nvneural::common_categories::Fused = "Fused"
	Category for layers that fuse multiple operations.
const char *const	nvneural::common_categories::General = "General"
	Category for layers that don't fit other categories.
const char *const	nvneural::common_categories::InputOutput = "Input/Output"
	Category for layers that interact with their host application.
const char *const	nvneural::common_categories::Shape = "Shape"
	Category for layers that adjust tensor shape.
const char *const	nvneural::common_categories::Training = "Training"
	Category for layers that represent training-only concepts.
const char *const	nvneural::common_categories::Upsampling = "Upsampling"
	Category for layers that upscale their input.

Detailed Description

Fundamental NvNeural data types are declared here.

Typedef Documentation

PrototypeRuntimeValidatorFunction

typedef bool(* nvneural::PrototypeRuntimeValidatorFunction) (const INetwork *pNetwork)

This function type represents a runtime validator.

It is used by a prototype layer to check that the network where the layer goes to satisfies the layer's specific requirements.

Parameters

pNetwork

A network pointer which a prototype layer is added to.

Returns

True if the network satisfies the layer's requirements, False otherwise.

Enumeration Type Documentation

ActivationFunctionId

enum nvneural::ActivationFunctionId : std::uint32_t

strong

Enumeration describing common activation functions.

You may place your own values in variables of this enumerated type, but to avoid conflicts all new values must be greater than CustomBase. Lower values are reserved for NVIDIA's use.

Activation function coefficients are described inside in-editor documentation, but are summarized in member descriptions.

Enumerator
None	No activation.
ReLU	ReLU ([0] = minimum value to return)
LeakyReLU	Leaky ReLU ([0] = slope)
LeakySigmoid	Leaky sigmoid ([0] = negative slope, [1] = positive slope)
LeakyTanh	Leaky tanh ([0] = negative slope, [1] = positive slope)
Tanh	Hyperbolic tangent.
Sigmoid	Sigmoid.
Softmax	Softmax.
Sine	Sine (also known as SIREN)
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

CompilationLevel

enum nvneural::CompilationLevel : std::uint16_t

strong

This enum represent kernel type contained by a prototype layer.

Enumerator
CompiledBinary	Example: SASS.
JitIntermediate	Examples: PTX, SPIR-V, or a compiled HLSL shader.
JitSource	Examples: CUDA C++, GLSL, or HLSL.

ImageSpace

enum nvneural::ImageSpace : std::uint32_t

strong

ImageSpace values refer to specific tensor representations of image data.

You may place your own values in variables of this enumerated type, but to avoid conflicts all new values must be greater than CustomBase. Lower values are reserved for NVIDIA's use.

Enumerator
Invalid	Invalid value used to guard against uninitialized variables.
RgbNormalized	RGB normalized to unit range; range is [0, 1].
RgbCentered	RGB centered around 0; range is [-1, 1].
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

LayerInputType

enum nvneural::LayerInputType : std::uint16_t

strong

This enum describes what type a particular input is for the layer.

A primary input is the typical case.

Secondary inputs are generally used if fixed weights are not desired. For instance, in a convolution layer, the kernel and bias inputs.

Another example, would be something like UpscaleConcatArbitrary, where one primary input is used for upscaling, and zero or more extra inputs are concatenated to the upscaled tensor. In this case, one Primary input and one PrimaryInfinite would need to be defined. Multiple PrimaryInfinite types would be ignored.

Enumerator
Primary	Most inputs are "primary" inputs.
PrimaryInfinite	Layers like concat and element_wise can take arbitrary numbers of inputs. Define one of these.
Secondary	Represents an input that is typically provided by the weights system.
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

MemorySemantic

enum nvneural::MemorySemantic : std::uint32_t

strong

Describes the intended purpose of allocated GPU memory.

Custom allocators may wish to segment allocators into distinct pools for performance or tracking reasons.

Enumerator
Unspecified	Generic allocation type, not tagged for any specific use.
Tensor	Memory used for layer tensors; typically larger allocations.
Weights	Memory used for weights data; typically long-lived.
Ephemeral	Memory used for short-lived scratch computations.
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

NetworkBackendId

enum nvneural::NetworkBackendId : std::uint32_t

strong

Enumeration describing common network backends.

You may place your own values in variables of this enumerated type, but to avoid conflicts all new values must be greater than CustomBase. Lower values are reserved for NVIDIA's use.

Enumerator
Invalid	Invalid value used to guard against uninitialized variables.
TestOnly	Only used in a testing environment.
Cpu	Pure CPU code, no GPU involvement (largely unused)
Cuda	CUDA/cuDNN backend.
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

NeuralResult

enum nvneural::NeuralResult : std::int32_t

strong

NeuralResult is a generic success/failure result type similar to COM HRESULT.

Enumerator
Success	Operation succeeded. Generic result.
Failure	Operation failed. Generic result.
Unsupported	Operation failed and will not pass on a retry (not implemented or logic error).

ParamType

enum nvneural::ParamType : std::uint16_t

strong

This enum describes what type a particular parameter.

New parameters, if needed, should not change existing numbers and should continue sequentially. There are also sanity checks that may need to be updated in xDescHostContainer.

xParamTypeDescs, in general, should not be modified without checking in the current code and taking in account there may be hosts/plugins that are still using the unmodified versions.

Enumerator
MaxParam	Should also be the next number for new non-custom types.
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

TensorDataLayout

enum nvneural::TensorDataLayout : std::uint16_t

strong

Enumeration describing common tensor data layouts.

You may place your own values in variables of this enumerated type, but to avoid conflicts all new values must be greater than CustomBase. Lower values are reserved for NVIDIA's use.

Enumerator
Invalid	Invalid value used to guard against uninitialized variables.
TestOnly	Only used in a testing environment.
Nchw	NCHW layout.
Nhwc	NHWC layout.
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

TensorDataType

enum nvneural::TensorDataType : std::uint16_t

strong

Enumeration describing common tensor element types.

You may place your own values in variables of this enumerated type, but to avoid conflicts all new values must be greater than CustomBase. Lower values are reserved for NVIDIA's use.

Enumerator
Invalid	Invalid value used to guard against uninitialized variables.
TestOnly	Only used in a testing environment.
Float	32-bit floating point elements (float)
Half	16-bit floating point elements (__half)
CompressedPng	NV Internal: Used to transit png like tensor between host and inference target.
UncompressedRgb	NV Internal: Used to send uncompressed R8G8B8 data between tools.
CustomBase	Custom entries in this enum should use IDs numerically higher than this value.

Variable Documentation

ApiVersion

const std::uint32_t nvneural::ApiVersion = 1338

API version set.

NVIDIA will increment the value of this constant when making backwards-incompatible changes to the framework. You should not mix binaries compiled against different version sets, as this may lead to crashes or memory corruption. Check for inequality only; do not try to find meaning in the specific value of this constant.