nvinfer1::IRotaryEmbeddingLayer Class Reference

Layer that implements Rotary Position Embedding (RoPE) (https://arxiv.org/abs/2104.09864). More...

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IRotaryEmbeddingLayer:

Public Member Functions
void	setInterleaved (bool interleaved) noexcept
	Set whether the input is in interleaved format, i.e., whether the 2-d vectors rotated are taken from adjacent 2 elements in the hidden dimension. The default value is false. More...
TRT_NODISCARD bool	getInterleaved () const noexcept
	Get whether the input is in interleaved format. The default value is false. More...
TRT_NODISCARD bool	setRotaryEmbeddingDim (int32_t rotaryEmbeddingDim) noexcept
	Set the number of hidden dimensions participating in RoPE. The default value is 0, representing H, i.e., all hidden dimensions in each head. Must be non-negative and even. More...
TRT_NODISCARD int32_t	getRotaryEmbeddingDim () const noexcept
	Get the number of hidden dimensions participating in RoPE. The default value is 0, representing H, i.e., all hidden dimensions in each head. More...
void	setInput (int32_t index, ITensor &tensor) noexcept
	Append or replace an input of this layer with a specific tensor. More...
Public Member Functions inherited from nvinfer1::ILayer
LayerType	getType () const noexcept
	Return the type of a layer. More...
void	setName (char const *name) noexcept
	Set the name of a layer. More...
char const *	getName () const noexcept
	Return the name of a layer. More...
int32_t	getNbInputs () const noexcept
	Get the number of inputs of a layer. More...
ITensor *	getInput (int32_t index) const noexcept
	Get the layer input corresponding to the given index. More...
int32_t	getNbOutputs () const noexcept
	Get the number of outputs of a layer. More...
ITensor *	getOutput (int32_t index) const noexcept
	Get the layer output corresponding to the given index. More...
void	setInput (int32_t index, ITensor &tensor) noexcept
	Replace an input of this layer with a specific tensor. More...
TRT_DEPRECATED void	setPrecision (DataType dataType) noexcept
	Set the preferred or required computational precision of this layer in a weakly-typed network. More...
DataType	getPrecision () const noexcept
	get the computational precision of this layer More...
TRT_DEPRECATED bool	precisionIsSet () const noexcept
	whether the computational precision has been set for this layer More...
TRT_DEPRECATED void	resetPrecision () noexcept
	reset the computational precision for this layer More...
TRT_DEPRECATED void	setOutputType (int32_t index, DataType dataType) noexcept
	Set the output type of this layer in a weakly-typed network. More...
DataType	getOutputType (int32_t index) const noexcept
	get the output type of this layer More...
TRT_DEPRECATED bool	outputTypeIsSet (int32_t index) const noexcept
	whether the output type has been set for this layer More...
TRT_DEPRECATED void	resetOutputType (int32_t index) noexcept
	reset the output type for this layer More...
void	setMetadata (char const *metadata) noexcept
	Set the metadata for this layer. More...
char const *	getMetadata () const noexcept
	Get the metadata of the layer. More...

Protected Member Functions
virtual	~IRotaryEmbeddingLayer () noexcept=default
Protected Member Functions inherited from nvinfer1::ILayer
virtual	~ILayer () noexcept=default
Protected Member Functions inherited from nvinfer1::INoCopy
	INoCopy ()=default
virtual	~INoCopy ()=default
	INoCopy (INoCopy const &other)=delete
INoCopy &	operator= (INoCopy const &other)=delete
	INoCopy (INoCopy &&other)=delete
INoCopy &	operator= (INoCopy &&other)=delete

Protected Attributes
apiv::VRotaryEmbeddingLayer *	mImpl
Protected Attributes inherited from nvinfer1::ILayer
apiv::VLayer *	mLayer

Detailed Description

Layer that implements Rotary Position Embedding (RoPE) (https://arxiv.org/abs/2104.09864).

Warning

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

~IRotaryEmbeddingLayer()

virtual nvinfer1::IRotaryEmbeddingLayer::~IRotaryEmbeddingLayer ( )

protectedvirtualdefaultnoexcept

Member Function Documentation

getInterleaved()

TRT_NODISCARD bool nvinfer1::IRotaryEmbeddingLayer::getInterleaved ( ) const

inlinenoexcept

Get whether the input is in interleaved format. The default value is false.

See also

setInterleaved

getRotaryEmbeddingDim()

TRT_NODISCARD int32_t nvinfer1::IRotaryEmbeddingLayer::getRotaryEmbeddingDim ( ) const

inlinenoexcept

Get the number of hidden dimensions participating in RoPE. The default value is 0, representing H, i.e., all hidden dimensions in each head.

See also

setRotaryEmbeddingDim

setInput()

void nvinfer1::ILayer::setInput ( int32_t  index, ITensor &  tensor  )

inlinenoexcept

Append or replace an input of this layer with a specific tensor.

Parameters

index	the index of the input to modify.
tensor	the new input tensor

The indices are as follows:

Input 0 is the input activation tensor. Input 1 is the cosine cache tensor. Input 2 is the sine cache tensor. Input 3 (optional) is the positionIds tensor, which is used for indexing into the cosine and sine caches.

setInterleaved()

void nvinfer1::IRotaryEmbeddingLayer::setInterleaved ( bool  interleaved )

inlinenoexcept

Set whether the input is in interleaved format, i.e., whether the 2-d vectors rotated are taken from adjacent 2 elements in the hidden dimension. The default value is false.

See also

getInterleaved

setRotaryEmbeddingDim()

TRT_NODISCARD bool nvinfer1::IRotaryEmbeddingLayer::setRotaryEmbeddingDim ( int32_t  rotaryEmbeddingDim )

inlinenoexcept

Set the number of hidden dimensions participating in RoPE. The default value is 0, representing H, i.e., all hidden dimensions in each head. Must be non-negative and even.

See also

getRotaryEmbeddingDim

Member Data Documentation

mImpl

apiv::VRotaryEmbeddingLayer* nvinfer1::IRotaryEmbeddingLayer::mImpl

protected

---

The documentation for this class was generated from the following file:

• NvInfer.h