ILayer is the base class for neural network layers. More...

#include <nvneural/LayerTypes.h>

Inheritance diagram for nvneural::ILayer:

Public Member Functions
virtual NetworkBackendId	backendId () const noexcept=0
	Returns the backend ID associated with this layer implementation. More...

virtual TensorDimension	dimensions () const noexcept=0
	Retrieves the dimensions of the layer's output tensor. More...

virtual NeuralResult	evaluateForward () noexcept=0
	Performs forward evaluation for this layer. More...

virtual NeuralResult	getInputLayers (ILayerList **ppInputLayers) const noexcept=0
	Retrieves the inputs for this layer. More...

virtual TensorDimension	internalDimensions () const noexcept=0
	Retrieves the dimensions of the layer's output tensor as allocated internally. More...

virtual NeuralResult	loadFromParameters (const IParameterNode *pParameters) noexcept=0
	Loads layer parameters from a serialized key-value representation. More...

virtual const char *	name () const noexcept=0
	Retrieves the layer name. More...

virtual NeuralResult	reshape () noexcept=0
	Initializes (or reinitializes) the layer implementation with the current set of parameters. More...

virtual const char *	serializedType () const noexcept=0
	Retrieves the layer type. More...

virtual NeuralResult	setInputLayer (std::size_t index, ILayer *pLayer) noexcept=0
	Sets an input layer by index. More...

virtual NeuralResult	setName (const char *pName) noexcept=0
	Sets the layer name. More...

virtual NeuralResult	setNetworkRuntime (INetworkRuntime *pNetworkRuntime) noexcept=0
	Informs the layer it has been attached to a new network. More...

virtual NeuralResult	setWeightsName (const char *pWeightsName) noexcept=0
	Sets the name used to identify this layer's weights. More...

virtual TensorDimension	stepping () const noexcept=0
	Returns the internal storage stride consumed by this layer implementation. More...

virtual std::size_t	tensorBufferSize () const noexcept=0
	Retrieve the size of the layer's output tensor buffer in bytes. More...

virtual TensorFormat	tensorFormat () const noexcept=0
	Returns the tensor format consumed by this layer implementation. More...

virtual std::size_t	tensorInternalBufferSize () const noexcept=0
	Retrieves the dimensions of the layer's output tensor as allocated internally. More...

virtual TensorDimension	weightsDimensions (const char *pWeightsName, WeightsQuery queryType) const noexcept=0
	Retrieves the tensor dimension of a layer's named weight input. More...

virtual const char *	weightsName () const noexcept=0
	Retrieves the name used to identify this layer's weights. More...

Activation functions
Layers may have attached activations in the network design. Activation function data is stored with the layer in order to support fused operations. Where possible, layer implementations should implement their own built-in activation functions as part of their operation kernels. This is significantly more efficient than a second kernel launch and full traversal of the layer's output tensor. When fusing launches in this way, do not call the INetworkRuntime::defaultForwardActivation function during inference. Only call that function when the activation function attached to your layer instance is one not already part of the layer inference kernel. Activation coefficients not explicitly assigned with setActivationCoefficient should be assumed to have default values of zero. Layers should allow arbitrary coefficient indices for composability with arbitrary activations, but may reject indices that would be nonsensical for their internal storage. Layers should allow at least the indices [0..3] to be stored in all cases.
virtual NeuralResult	setActivationFunction (ActivationFunctionId activationFunction) noexcept=0
	Sets the activation function attached to the layer. More...

virtual ActivationFunctionId	activationFunction () const noexcept=0
	Retrieves the activation function attached to this layer. More...

virtual NeuralResult	setActivationCoefficient (std::size_t coefficientIndex, float value) noexcept=0
	Sets an activation coefficient. More...

virtual float	activationCoefficient (std::size_t coefficientIndex) const noexcept=0
	Retrieves the activation coefficient for the specified index. More...

Tensor memory management flags
virtual NeuralResult	setPermanent (bool permanent) noexcept=0
	Sets or clears the "permanent" flag on a layer's output tensor. More...

virtual bool	isPermanent () const noexcept=0
	Returns the current status of the "permanent" flag. More...

virtual NeuralResult	setAffected (bool affected) noexcept=0
	Sets or clears the "affected" flag on a layer's output tensor. More...

virtual bool	isAffected () const noexcept=0
	Returns the current status of the "affected" flag. More...

Data retrieval
virtual NeuralResult	getData (void *ppOut, TensorFormat format, const ILayer pRequestingLayer) noexcept=0
	Retrieves device-side memory for the layer's output.

virtual NeuralResult	getConstData (const void *ppOut, TensorFormat format, const ILayer pRequestingLayer) const noexcept=0
	Retrieves read-only device-side memory for the layer's output.

virtual NeuralResult	getCpuConstData (void pOutBuffer, size_t bufferByteCount, size_t pBytesCopied, TensorFormat format) const noexcept=0
	Retrieves read-only CPU-side memory for the layer's output. More...

Public Member Functions inherited from nvneural::IRefObject
virtual RefCount	addRef () const noexcept=0
	Increments the object's reference count. More...

virtual const void *	queryInterface (TypeId interface) const noexcept=0
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

virtual void *	queryInterface (TypeId interface) noexcept=0
	Retrieves a new object interface pointer. More...

virtual RefCount	release () const noexcept=0
	Decrements the object's reference count and destroy the object if the reference count reaches zero. More...

Static Public Attributes
static const IRefObject::TypeId	typeID = 0xcd07ffbfd74d9b11ul
	Interface TypeId for InterfaceOf purposes.

Static Public Attributes inherited from nvneural::IRefObject
static const TypeId	typeID = 0x14ecc3f9de638e1dul
	Interface TypeId for InterfaceOf purposes.

Additional Inherited Members
Public Types inherited from nvneural::IRefObject
using	RefCount = std::uint32_t
	Typedef used to track the number of active references to an object.

using	TypeId = std::uint64_t
	Every interface must define a unique TypeId. This should be randomized.

Protected Member Functions inherited from nvneural::IRefObject
virtual	~IRefObject ()=default
	A protected destructor prevents accidental stack-allocation of IRefObjects or use with other smart pointer classes like std::unique_ptr.

Detailed Description

ILayer is the base class for neural network layers.

Member Function Documentation

◆ activationCoefficient()

virtual float nvneural::ILayer::activationCoefficient ( std::size_t coefficientIndex ) const

pure virtualnoexcept

Retrieves the activation coefficient for the specified index.

The meaning of this value is activation-specific. If no setActivationCoefficient call was previously issued for this index, the layer should return a default value of zero.

Parameters

coefficientIndex Coefficient index

Implemented in nvneural::BaseLayer.

◆ activationFunction()

virtual ActivationFunctionId nvneural::ILayer::activationFunction ( ) const

pure virtualnoexcept

Retrieves the activation function attached to this layer.

Should return the most recent value passed to setActivationFunction.

Implemented in nvneural::BaseLayer.

◆ backendId()

virtual NetworkBackendId nvneural::ILayer::backendId ( ) const

pure virtualnoexcept

Returns the backend ID associated with this layer implementation.

Networks will reshape tensors as necessary in order to match the return value.

Implemented in nvneural::BaseLayer.

◆ dimensions()

virtual TensorDimension nvneural::ILayer::dimensions ( ) const

pure virtualnoexcept

Retrieves the dimensions of the layer's output tensor.

Should not change except in response to a reshape event.

◆ evaluateForward()

virtual NeuralResult nvneural::ILayer::evaluateForward ( )

pure virtualnoexcept

Performs forward evaluation for this layer.

If the layer does not provide a pre-fused implementation of the selected activation function, it MUST call INetworkRuntime::defaultForwardActivation(this).

◆ getCpuConstData()

virtual NeuralResult nvneural::ILayer::getCpuConstData	(	void *	pOutBuffer,
		size_t	bufferByteCount,
		size_t *	pBytesCopied,
		TensorFormat	format
	)		const

pure virtualnoexcept

Retrieves read-only CPU-side memory for the layer's output.

This function is intended for debugging purposes. Element count of the buffer should be dimensions().elementCount(), multiplied by the size of the desired element type.

To check buffer sizes without preallocating memory, pass nullptr in pOutBuffer and zero in bufferByteCount. pBytesCopied will then receive the optimal size for pOutBuffer. Other uses of nullptr are considered errors.

pBytesCopied is not modified if this function fails.

Parameters

pOutBuffer	Memory region receiving the layer output data
bufferByteCount	Size of pOutBuffer in bytes
pBytesCopied	Optional output variable receiving the number of bytes copied into pOutBuffer
format	Format for the output buffer

Implemented in nvneural::BaseLayer.

◆ getInputLayers()

virtual NeuralResult nvneural::ILayer::getInputLayers ( ILayerList ** ppInputLayers ) const

pure virtualnoexcept

Retrieves the inputs for this layer.

Parameters

ppInputLayers Pointer receiving a reference to an ILayerList object.

Layer may create a new list or return an incremented reference to a cached object; this is an implementation detail.

In either case, caller is responsible for releasing its reference when done with *ppInputLayers.

Implemented in nvneural::BaseLayer.

◆ internalDimensions()

virtual TensorDimension nvneural::ILayer::internalDimensions ( ) const

pure virtualnoexcept

Retrieves the dimensions of the layer's output tensor as allocated internally.

Internal allocations may have additional padding or alignment restrictions.

See also: stepping()

Implemented in nvneural::BaseLayer.

◆ isAffected()

virtual bool nvneural::ILayer::isAffected ( ) const

pure virtualnoexcept

Returns the current status of the "affected" flag.

By default, layers should be "affected" after initialization.

Implemented in nvneural::BaseLayer.

◆ isPermanent()

virtual bool nvneural::ILayer::isPermanent ( ) const

pure virtualnoexcept

Returns the current status of the "permanent" flag.

By default, layers should not be "permanent."

Implemented in nvneural::BaseLayer.

◆ loadFromParameters()

virtual NeuralResult nvneural::ILayer::loadFromParameters ( const IParameterNode * pParameters )

pure virtualnoexcept

Loads layer parameters from a serialized key-value representation.

Tools such as ConverenceNG call this during network construction. Deployed applications may use this function themselves (often with a container class such as StringMapParameterNode) or may use layer-specific object interfaces; either approach is valid.

Implemented in nvneural::BaseLayer.

◆ name()

virtual const char* nvneural::ILayer::name ( ) const

pure virtualnoexcept

Retrieves the layer name.

Returns: Layer name; pointer must be valid until object is destroyed or setName is called.

Implemented in nvneural::BaseLayer.

◆ reshape()

virtual NeuralResult nvneural::ILayer::reshape ( )

pure virtualnoexcept

Initializes (or reinitializes) the layer implementation with the current set of parameters.

It is safe to call this multiple times, and may be necessary as parameters change.

◆ serializedType()

virtual const char* nvneural::ILayer::serializedType ( ) const

pure virtualnoexcept

Retrieves the layer type.

Returns: Layer type; pointer must be a non null string.

◆ setActivationCoefficient()

virtual NeuralResult nvneural::ILayer::setActivationCoefficient	(	std::size_t	coefficientIndex,
		float	value
	)

pure virtualnoexcept

Sets an activation coefficient.

The meaning of this value is activation-specific.

Parameters

coefficientIndex	Coefficient index
value	Value to store

Implemented in nvneural::BaseLayer.

◆ setActivationFunction()

virtual NeuralResult nvneural::ILayer::setActivationFunction ( ActivationFunctionId activationFunction )

pure virtualnoexcept

Sets the activation function attached to the layer.

Repeated calls to this function replace the activation rather than stacking them.

Parameters

activationFunction New activation function ID

Implemented in nvneural::BaseLayer.

◆ setAffected()

virtual NeuralResult nvneural::ILayer::setAffected ( bool affected )

pure virtualnoexcept

Sets or clears the "affected" flag on a layer's output tensor.

Layers that are "affected" have pending computation work and will be reevaluated as necessary during inference. Networks will clear the "affected" flag after successful inference.

Layers should mark themselves as "affected" in response to parameter changes that would be visible during inference. Examples of such changes include loadFromParameters and the IRuntimeOptionsLayer::setRuntimeOptionValue callback.

Parameters

affected New value for the "affected" flag

Implemented in nvneural::BaseLayer.

◆ setInputLayer()

virtual NeuralResult nvneural::ILayer::setInputLayer	(	std::size_t	index,
		ILayer *	pLayer
	)

pure virtualnoexcept

Sets an input layer by index.

Input indices are explicitly allowed to be sparse in order to support optional inputs; usually such inputs are used to provide weights from other parts of the graph instead of going out to IWeightsLoader.

Parameters

index	Index of the input to assign
pLayer	Layer to assign as an input

Implemented in nvneural::BaseLayer.

◆ setName()

virtual NeuralResult nvneural::ILayer::setName ( const char * pName )

pure virtualnoexcept

Sets the layer name.

Layer names are unique within a network.

Parameters

pName New layer name; layer must copy this data.

Implemented in nvneural::BaseLayer.

◆ setNetworkRuntime()

virtual NeuralResult nvneural::ILayer::setNetworkRuntime ( INetworkRuntime * pNetworkRuntime )

pure virtualnoexcept

Informs the layer it has been attached to a new network.

INetwork::pushLayer implementations will call this function for you; host applications do not normally need to call this function.

Parameters

pNetworkRuntime New owning network

Implemented in nvneural::BaseLayer.

◆ setPermanent()

virtual NeuralResult nvneural::ILayer::setPermanent ( bool permanent )

pure virtualnoexcept

Sets or clears the "permanent" flag on a layer's output tensor.

Layers that are "permanent" are kept resident in memory and their tensor buffers are not reused. Setting too many layers as "permanent" will have negative effects on memory utilization.

Parameters

permanent New value for the "permanent" flag

Implemented in nvneural::BaseLayer.

◆ setWeightsName()

virtual NeuralResult nvneural::ILayer::setWeightsName ( const char * pWeightsName )

pure virtualnoexcept

Sets the name used to identify this layer's weights.

Parameters

pWeightsName New layer name to use for weights instead of name()

Implemented in nvneural::BaseLayer.

◆ stepping()

virtual TensorDimension nvneural::ILayer::stepping ( ) const

pure virtualnoexcept

Returns the internal storage stride consumed by this layer implementation.

Output tensors are padded to be multiples of this value.

Implemented in nvneural::BaseLayer.

◆ tensorBufferSize()

virtual std::size_t nvneural::ILayer::tensorBufferSize ( ) const

pure virtualnoexcept

Retrieve the size of the layer's output tensor buffer in bytes.

This result is defined as the number of elements in dimensions() multiplied by the size of an individual element (e.g., 4 for FP32).

Implemented in nvneural::BaseLayer.

◆ tensorFormat()

virtual TensorFormat nvneural::ILayer::tensorFormat ( ) const

pure virtualnoexcept

Returns the tensor format consumed by this layer implementation.

Networks will reshape tensors as necessary in order to match the return value.

Implemented in nvneural::BaseLayer.

◆ tensorInternalBufferSize()

virtual std::size_t nvneural::ILayer::tensorInternalBufferSize ( ) const

pure virtualnoexcept

Retrieves the dimensions of the layer's output tensor as allocated internally.

Internal allocations may have additional padding or alignment restrictions. The result is defined as the number of elements in internalDimensions() multiplied by the size of an individual element (e.g., 4 for FP32).

Implemented in nvneural::BaseLayer.

◆ weightsDimensions()

virtual TensorDimension nvneural::ILayer::weightsDimensions	(	const char *	pWeightsName,
		WeightsQuery	queryType
	)		const

pure virtualnoexcept

Retrieves the tensor dimension of a layer's named weight input.

Parameters

pWeightsName	Name of a weights object being queried
queryType	Type of dimensions being queried; see WeightsQuery for details

Implemented in nvneural::BaseLayer.

◆ weightsName()

virtual const char* nvneural::ILayer::weightsName ( ) const

pure virtualnoexcept

Retrieves the name used to identify this layer's weights.

By default, this should be the name assigned with setName. Alternate weights names allow layer aliasing and namespacing within subnetworks.

Implemented in nvneural::BaseLayer.

The documentation for this class was generated from the following file:

Core/Inc/nvneural/LayerTypes.h

Public Member Functions

Static Public Attributes

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ activationCoefficient()

◆ activationFunction()

◆ backendId()

◆ dimensions()

◆ evaluateForward()

◆ getCpuConstData()

◆ getInputLayers()

◆ internalDimensions()

◆ isAffected()

◆ isPermanent()

◆ loadFromParameters()

◆ name()

◆ reshape()

◆ serializedType()

◆ setActivationCoefficient()

◆ setActivationFunction()

◆ setAffected()

◆ setInputLayer()

◆ setName()

◆ setNetworkRuntime()

◆ setPermanent()

◆ setWeightsName()

◆ stepping()

◆ tensorBufferSize()

◆ tensorFormat()

◆ tensorInternalBufferSize()

◆ weightsDimensions()

◆ weightsName()