INetworkRuntime is a subset of the basic network interface that is accessible from layer classes during network evaluation. More...

#include <nvneural/CoreTypes.h>

Inheritance diagram for nvneural::INetworkRuntime:

Public Member Functions
virtual NetworkBackendId	defaultBackendId () const noexcept=0
	Retrieves the default backend ID used by the network.

virtual NeuralResult	defaultForwardActivation (ILayer *pLayer) const noexcept=0
	Perform generic forward activation. More...

virtual TensorFormat	defaultTensorFormat () const noexcept=0
	Retrieves the default tensor format used by the network.

virtual const INetworkBackend *	getBackend (NetworkBackendId backendId) const noexcept=0
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

virtual INetworkBackend *	getBackend (NetworkBackendId backendId) noexcept=0
	Retrieves an installed network backend. More...

virtual void *	getInternalTensor (const ILayer *pLayer, NetworkBackendId backendId) noexcept=0
	Retrieves internal tensor data for a layer. More...

virtual TensorDimension	getLayerInternalDimensions (const ILayer *pLayer) const noexcept=0
	Retrieves the internal dimensions of a layer. More...

virtual void *	getTensor (const ILayer pDataLayer, const ILayer pRequestingLayer, TensorFormat format) noexcept=0
	Retrieves tensor data from one layer on behalf of another. More...

virtual NeuralResult	getWeightsDimensionForLayer (TensorDimension pDimensionsOut, const ILayer pLayer, const char *pWeightsName, NetworkBackendId backendId, TensorFormat format) noexcept=0
	Retrieves the dimensions of specific weights data for a layer. More...

virtual const void *	getWeightsForLayer (const ILayer pLayer, const char pWeightsName, NetworkBackendId backendId, TensorFormat format) noexcept=0
	Retrieves specific weights data for a layer. More...

virtual NeuralResult	mapLayerAsWeightsData (const ILayer pDataLayer, const ILayer pWeightsLayer, const char *pWeightsName) noexcept=0
	Maps a layer as a source of weights data for another layer. More...

virtual const char *	networkName () const noexcept=0
	Retrieves the name of this network as a UTF-8 string.

virtual NeuralResult	reshapeLater (ILayer *pLayer) noexcept=0
	Queues a layer for later reshape. More...

virtual NeuralResult	setAffectedForward (ILayer *pLayer) noexcept=0
	Marks a layer and its descendants (those layers indirectly depending on it) as "affected.". More...

virtual IWeightsLoader *	weightsLoader () const noexcept=0
	Retrieves the IWeightsLoader implementation for this network.

Layer enumeration and retrieval
virtual size_t	layerCount () const noexcept=0
	Gets the current number of layers in the network.

virtual ILayer *	getLayerByName (const char *pLayerName) const noexcept=0
	Retrieves a layer by name. More...

virtual NeuralResult	getAllLayers (ILayerList **ppListOut) const noexcept=0
	Retrieves a list of layers present in the network. More...

virtual NeuralResult	getInputLayers (ILayerList **ppListOut) const noexcept=0
	Retrieves a list of input layers present in the network. More...

virtual NeuralResult	getOutputLayers (ILayerList **ppListOut) const noexcept=0
	Retrieves a list of output layers present in the network. More...

virtual NeuralResult	getDependentLayers (ILayerList *ppListOut, const ILayer pLayer) const noexcept=0
	Retrieves a list of layers indirectly depending on pLayer. More...

virtual NeuralResult	getAcceptorLayers (ILayerList *ppListOut, const ILayer pLayer) const noexcept=0
	Retrieves a list of layers directly depending on pLayer. More...

virtual NeuralResult	getInfluenceLayers (ILayerList *ppListOut, const ILayer pLayer) const noexcept=0
	Retrieves a list of layers pLayer indirectly depends on. More...

Public Member Functions inherited from nvneural::IRefObject
virtual RefCount	addRef () const noexcept=0
	Increments the object's reference count. More...

virtual const void *	queryInterface (TypeId interface) const noexcept=0
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

virtual void *	queryInterface (TypeId interface) noexcept=0
	Retrieves a new object interface pointer. More...

virtual RefCount	release () const noexcept=0
	Decrements the object's reference count and destroy the object if the reference count reaches zero. More...

Static Public Attributes
static const IRefObject::TypeId	typeID = 0x53191c073f990992ul
	Interface TypeId for InterfaceOf purposes.

Static Public Attributes inherited from nvneural::IRefObject
static const TypeId	typeID = 0x14ecc3f9de638e1dul
	Interface TypeId for InterfaceOf purposes.

Additional Inherited Members
Public Types inherited from nvneural::IRefObject
using	RefCount = std::uint32_t
	Typedef used to track the number of active references to an object.

using	TypeId = std::uint64_t
	Every interface must define a unique TypeId. This should be randomized.

Protected Member Functions inherited from nvneural::IRefObject
virtual	~IRefObject ()=default
	A protected destructor prevents accidental stack-allocation of IRefObjects or use with other smart pointer classes like std::unique_ptr.

Detailed Description

INetworkRuntime is a subset of the basic network interface that is accessible from layer classes during network evaluation.

Member Function Documentation

◆ defaultForwardActivation()

virtual NeuralResult nvneural::INetworkRuntime::defaultForwardActivation ( ILayer * pLayer ) const

pure virtualnoexcept

Perform generic forward activation.

Layer classes should call this for any activations they do not support with their own pre-fused implementations. Modifies layer tensors directly.

Parameters

pLayer Layer to perform activation step for

◆ getAcceptorLayers()

virtual NeuralResult nvneural::INetworkRuntime::getAcceptorLayers	(	ILayerList **	ppListOut,
		const ILayer *	pLayer
	)		const

pure virtualnoexcept

Retrieves a list of layers directly depending on pLayer.

Parameters

ppListOut	Receives a reference to a new ILayerList containing layers depending directly on pLayer. Does not contain pLayer.
pLayer	Layer to query.

◆ getAllLayers()

virtual NeuralResult nvneural::INetworkRuntime::getAllLayers ( ILayerList ** ppListOut ) const

pure virtualnoexcept

Retrieves a list of layers present in the network.

The collection is not sorted.

Parameters

ppListOut Receives a reference to a new ILayerList object. Caller must release it.

◆ getBackend()

virtual INetworkBackend* nvneural::INetworkRuntime::getBackend ( NetworkBackendId backendId )

pure virtualnoexcept

Retrieves an installed network backend.

Parameters

backendId Network backend identifier to retrieve

Returns: The INetworkBackend for that interface, or nullptr if no such backend was found.

◆ getDependentLayers()

virtual NeuralResult nvneural::INetworkRuntime::getDependentLayers	(	ILayerList **	ppListOut,
		const ILayer *	pLayer
	)		const

pure virtualnoexcept

Retrieves a list of layers indirectly depending on pLayer.

Parameters

ppListOut	Receives a reference to a new ILayerList containing pLayer and other indirect dependencies.
pLayer	Layer to query.

◆ getInfluenceLayers()

virtual NeuralResult nvneural::INetworkRuntime::getInfluenceLayers	(	ILayerList **	ppListOut,
		const ILayer *	pLayer
	)		const

pure virtualnoexcept

Retrieves a list of layers pLayer indirectly depends on.

Parameters

ppListOut	Receives a reference to a new ILayerList containing layers pLayer relies on, including itself.
pLayer	Layer to query.

◆ getInputLayers()

virtual NeuralResult nvneural::INetworkRuntime::getInputLayers ( ILayerList ** ppListOut ) const

pure virtualnoexcept

Retrieves a list of input layers present in the network.

Input layers implement the IInputLayer interface.

The collection is not sorted.

Parameters

ppListOut Receives a reference to a new ILayerList object. Caller must release it.

◆ getInternalTensor()

virtual void* nvneural::INetworkRuntime::getInternalTensor	(	const ILayer *	pLayer,
		NetworkBackendId	backendId
	)

pure virtualnoexcept

Retrieves internal tensor data for a layer.

Returned value is a pointer in the requested backend's address space. For example, this would be a device void* in CUDA.

Parameters

pLayer	Layer containing the requested data
backendId	Backend for the destination

Only INetworkBackend objects will typically need to use this function.

◆ getLayerByName()

virtual ILayer* nvneural::INetworkRuntime::getLayerByName ( const char * pLayerName ) const

pure virtualnoexcept

Retrieves a layer by name.

Parameters

pLayerName Layer name to retrieve

Returns: The ILayer by that name, or nullptr if no such layer was found.

◆ getLayerInternalDimensions()

virtual TensorDimension nvneural::INetworkRuntime::getLayerInternalDimensions ( const ILayer * pLayer ) const

pure virtualnoexcept

Retrieves the internal dimensions of a layer.

In the general case this method is equivalent to ILayer::internalDimensions but it also handles the case of IBypassLayers, if the layer will be bypassed the internal dimensions returned will be the aliased layer ones.

Parameters

pLayer Layer associated with the requested data

Returns: The internal dimensions of the layer

◆ getOutputLayers()

virtual NeuralResult nvneural::INetworkRuntime::getOutputLayers ( ILayerList ** ppListOut ) const

pure virtualnoexcept

Retrieves a list of output layers present in the network.

Output layers are not consumed as inputs by other layers. This is not the same as the "Output" layer type; those layers are trimmed during network construction to reduce memory copies.

Parameters

ppListOut Receives a reference to a new ILayerList object. Caller must release it.

◆ getTensor()

virtual void* nvneural::INetworkRuntime::getTensor	(	const ILayer *	pDataLayer,
		const ILayer *	pRequestingLayer,
		TensorFormat	format
	)

pure virtualnoexcept

Retrieves tensor data from one layer on behalf of another.

Returned value is a pointer in the requesting layer's backend's address space. For example, this would be a device void* in CUDA.

Parameters

pDataLayer	Layer containing the requested data
pRequestingLayer	Layer requesting the data
format	Tensor format to deliver data in

◆ getWeightsDimensionForLayer()

virtual NeuralResult nvneural::INetworkRuntime::getWeightsDimensionForLayer	(	TensorDimension *	pDimensionsOut,
		const ILayer *	pLayer,
		const char *	pWeightsName,
		NetworkBackendId	backendId,
		TensorFormat	format
	)

pure virtualnoexcept

Retrieves the dimensions of specific weights data for a layer.

Parameters

pDimensionsOut	Output pointer to the requested dimensions
pLayer	Layer associated with the requested data
pWeightsName	Name of the weights data
backendId	Backend ID to query
format	Tensor format to query

◆ getWeightsForLayer()

virtual const void* nvneural::INetworkRuntime::getWeightsForLayer	(	const ILayer *	pLayer,
		const char *	pWeightsName,
		NetworkBackendId	backendId,
		TensorFormat	format
	)

pure virtualnoexcept

Retrieves specific weights data for a layer.

Returned value is a pointer in the requested backend's address space. For example, this would be a device void* in CUDA.

Returned value is also in the format requested.

Parameters

pLayer	Layer associated with the requested data
pWeightsName	Name of the weights data
backendId	Backend ID to deliver data using
format	Tensor format to deliver data in

◆ mapLayerAsWeightsData()

virtual NeuralResult nvneural::INetworkRuntime::mapLayerAsWeightsData	(	const ILayer *	pDataLayer,
		const ILayer *	pWeightsLayer,
		const char *	pWeightsName
	)

pure virtualnoexcept

Maps a layer as a source of weights data for another layer.

Calls to getWeightsForLayer from pDataLayer requesting pWeightsName will return the output tensor of pWeightsLayer rather than dispatching to the network's weights loader.

To unmap a layer from this system, pass nullptr as pWeightsLayer.

Parameters

pDataLayer	Layer that will request weights
pWeightsLayer	Layer that will provide weights
pWeightsName	Name to use for the weights data

◆ reshapeLater()

virtual NeuralResult nvneural::INetworkRuntime::reshapeLater ( ILayer * pLayer )

pure virtualnoexcept

Queues a layer for later reshape.

The reshape operation will happen prior to the next inference pass.

◆ setAffectedForward()

virtual NeuralResult nvneural::INetworkRuntime::setAffectedForward ( ILayer * pLayer )

pure virtualnoexcept

Marks a layer and its descendants (those layers indirectly depending on it) as "affected.".

This is equivalent to calling ILayer::setAffected on everything returned by getDependentLayers. It is provided to layer classes as a convenience function.

Parameters

pLayer Layer to mark

The documentation for this class was generated from the following file:

Core/Inc/nvneural/CoreTypes.h

Public Member Functions

Static Public Attributes

Additional Inherited Members

Detailed Description

Member Function Documentation

◆ defaultForwardActivation()

◆ getAcceptorLayers()

◆ getAllLayers()

◆ getBackend()

◆ getDependentLayers()

◆ getInfluenceLayers()

◆ getInputLayers()

◆ getInternalTensor()

◆ getLayerByName()

◆ getLayerInternalDimensions()

◆ getOutputLayers()

◆ getTensor()

◆ getWeightsDimensionForLayer()

◆ getWeightsForLayer()

◆ mapLayerAsWeightsData()

◆ reshapeLater()

◆ setAffectedForward()