NVRC Server API

NVIDIA Docs Hub NVIDIA Isaac
NVIDIA Isaac SDK 2021.1 NVRC Server API

This is a reference page for the server API used within the Isaac RC module in the Isaac SDK app.

NVRC Common API

Overview

Data Structures

Structure	Description
nvrcCameraProperties	Various details associated with a camera
nvrcData	A message containing arbitrary data
nvrcDataHeader	The metadata for a message
nvrcDataStreamParams	Reserved for future use.
nvrcImageParams	Metadata for a video frame
nvrcInputData	An input event sent over the data stream
nvrcLatencyData	Summary of latency data collected during a measurement period (since the last latency measurement callback)
nvrcMosaicViewport	Various details associated with one section of a mosaic image
nvrcServerConfig	Platform and streaming-specific information about the server
nvrcStreamingParams	Various details associated with multiple video streams or a mosaic stream, as well as a data stream
nvrcUserInputEvent	The contents of the user input event
nvrcVideoFrame	A frame of video containing metadata and a buffer for the data of the frame itself
nvrcVideoFrameGroup	A collection of video frames
nvrcVideoStreamParams	Various details associated with a video stream

Macros

Macro	Value	Description
NVRC_MAX_DATA_LABEL_LENGTH	50	The maximum length of the label in a data message.
NVRC_MAX_STREAM_NAME_LENGTH	50	Maximum length of the name of a stream or viewport.
NVRC_MAX_VIDEO_FRAME_GROUP_COUNT	4	The maximum number of frames in a frame group.
NVRC_MAX_VIDEO_STREAMS	4	Maximum number of video streams in a nvrcStreamingParams.

Typedefs

Typedef	Description
void (* nvrcLatencyMeasurementCallback)
uint64_t nvrcTime_t	The timestamp for a video frame or data message

Enumerations

Enum	Description
nvrcConnectionStatus	The status of a client connection
nvrcImageFormat	The organization of color channels in the frame buffer
nvrcImageType	A value that specifies whether the image is stored in a CPU buffer, or as either a CUDA or NVMEDIA image on GPU
nvrcInputType	The type of the input event
nvrcProjectionType	A value that specifies how the image of a camera is projected
nvrcStatus	The error return value from most NVRC functions
nvrcStreamType	The type of stream that the latency API is dealing with
nvrcSynchronizationType	The frequency control for data messages

Data Structures

nvrcCameraProperties

Various details associated with a camera. Definition is at line 62 of file config.h.

Data Fields

Data Type	Parameter	Description
float	bwPoly[5]	Backward-polynomial coefficients for a fisheye camera
float	opticalCenter[2]	The position of the optical center in pixels, laid out in the array as [x, y]
nvrcProjectionType	projectionType
float	rotation[4]	The quaternion rotation of the camera relative to the forward direction of the vehicle, laid out in the array as [w, x, y, z]
float	translation[3]	The location of the camera relative to the origin of the vehicle in meters, laid out in the array as [x, y, z]

nvrcData

A message containing arbitrary data. The definition is at line 183 of file frame.h.

Data Type	Parameter	Description
nvrcDataHeader	header	The metadata of the message
void *	ptr	A CPU buffer containing the data of the message

nvrcDataHeader

The metadata for a message. The definition is at line 153 of file frame.h.

Data Type	Parameter	Description
uint32_t	group_id	Reserved for future use.
char	label[NVRC_MAX_DATA_LABEL_LENGTH]	An arbitrary label. Can be used to differentiate between types of messages.
size_t	size	The number of bytes owned by the message from the beginning of the pointer provided in the parent nvrcData.
nvrcTime_t	timestamp_us	Epoc time in microseconds, typically just before the message is given to NVRC. This figure is used to measure latency and synchronize with video, so be deliberate about when this timestamp is recorded.
uint32_t	video_stream_id	Reserved for future use.

nvrcDataStreamParams

Reserved for future use. The definition is at line 126 of file config.h.

nvrcImageParams

The metadata for a video frame. The definition is at line 80 of file frame.h.

Data Type	Parameter	Description
nvrcImageFormat	format	The format of the frame buffer
uint32_t	height	The number of pixels down in the image
uint32_t	pitch	The number of bytes between the start of consecutive rows of the image
nvrcImageType	type	The type of the frame buffer
uint32_t	width	The number of pixels across in the image

nvrcInputData

An input event sent over the data stream. The definition is at line 74 of file input.h.

Data Type	Parameter	Description
union InputUnion	inputUnion
nvrcTime_t	timestamp_us	A timestamp set by the sender of the input event
nvrcInputType	type	The type corresponding to the alternative to choose in the inputUnion

nvrcLatencyData

The summary of latency data collected during a measurement period (since the last latency measurement callback).

Latencies are based on the timestamps present in the received packets, the system epoc time in microseconds, and some synchronization between the clocks of this node and the sender. If the packets are from some pre-recorded dataset, or if the timestamps are not formatted as epoc time in microseconds, this latency data will likely be nonsensical.

The definition is at line 65 of file latency.h.

Data Type	Parameter	Description
float	average_us	The average latency in microseconds of the packets received during the measurement period
nvrcTime_t_	max_us	The maximum latency in microseconds of the packets received during the measurement period
nvrcTime_t_	min_us	The minimum latency in microseconds of the packets received during the measurement period
uint32_t	num_packets	The number of packets received by this node during the measurement period
float	std_dev_us	The standard deviation of the latency in microseconds of the packets received during the measurement period
nvrcTime_t_	time_measured_us	Microseconds elapsed during the measurement period

nvrcMosaicViewport

Various details associated with one section of a mosaic image.

The definition is at line 134 of the file config.h.

Data Type	Parameter	Description
uint32_t	height	The number of pixels down in the images produced
char	name[NVRC_MAX_STREAM_NAME_LENGTH]	The name of the viewport (e.g. the name of the sensor that produces it)
uint32_t	width	The number of pixels across in the images produced
uint32_t	x	The number of pixels to shift the viewport right
uint32_t	y	The number of pixels to shift the viewport down

nvrcServerConfig

Platform and streaming-specific information about the server. The server uses this data to send a default or available config to the client, and the client responds with their desired config.

The definition is at line 210 of the file config.h.

Data Type	Parameter	Description
const char *	platformConfig	A JSON string describing the server platform config
nvrcStreamingParams	streamingParams	An object describing the server streaming config

nvrcStreamingParams

Details associated with multiple video streams or a mosaic stream, as well as a data stream.

The definition is at line 168 of the file config.h.

Data Type	Parameter	Description
nvrcDataStreamParams	dataStream	The data stream managed by the server
bool	mosaic	Specifies whether the server is in mosaic mode: There should only be one videoStream specified, which streams the composition of all viewports into a mosaic, and each stream specifies which viewport to use.
uint32_t	mosaicViewportCount	The number of items from the beginning of the mosaicViewports array that are valid.
nvrcMosaicViewport	mosaicViewports[NVRC_MAX_VIDEO_STREAMS]	One or more mosaic viewports managed by the server
uint32_t	videoStreamCount	The number of valid items from the beginning of the videoStreams array
nvrcVideoStreamParams	videoStreams[NVRC_MAX_VIDEO_STREAMS]	One or more video streams managed by the server

nvrcUserInputEvent

The content of a user input event.

The definition at line 58 of the file input.h.

Data Type	Parameter	Description
uint8_t *	data	The CPU buffer that contains the content of the user input event
uint32_t	size	The number of bytes from the beginning of the data buffer that represent the content of the user input event

nvrcVideoFrame

A frame of video containing some metadata and a buffer for the data of the frame itself.

The definition is at line 106 of the file frame.h.

Data Type	Parameter	Description
nvrcImageParams	params	Specifies how the data in the frame is organized.
void *	ptr	A buffer containing the frame data. The location of the buffer should correspond to the type specified in `params`.
uint32_t	stream_id	There may be multiple streams that can send data in parallel. Video sent on stream 0 will be received on stream 0.
nvrcTime_t	timestamp_us	The epoc time in microseconds: typically just before the frame is given to NVRC. Because this value is used to measure latency and synchronize with data, ensure that this timestamp is recorded at the correct moment.

nvrcVideoFrameGroup

A collection of video frames.

The definition is at line 137 of the file frame.h.

Data Type	Parameter	Description
uint32_t	count	The number of frames used in the frames array
nvrcVideoFrame	frames[NVRC_MAX_VIDEO_FRAME_GROUP_COUNT]	The frames in the group

nvrcVideoStreamParams

Various details associated with a video stream.

The definition is at line 97 of the file config.h.

Data Type	Parameter	Description
nvrcCameraProperties	cameraProperties	The properties of the camera producing the stream
uint32_t	height	The number of pixels down in the images produced
uint32_t	maxBitrate	The maximum bits per second of the stream
char	name[NVRC_MAX_STREAM_NAME_LENGTH]	The name of the stream; for instance, the name of the sensor that produces it
float	refreshRate	The frames per second of the stream
uint32_t	width	The number of pixels across in the images produced

Enumerations

nvrcConnectionStatus

The status of a client connection.

Enumerator	Value	Description
NVRC_CLIENT_CONNECTED	0
NVRC_CLIENT_RECONNECTING	1
NVRC_CLIENT_DISCONNECTED	2

nvrcImageFormat

The organization of color channels in the frame buffer.

Enumerator	Value	Description
NVRC_IMAGE_RGBA	0	Non-planar; 1 byte per channel; red channel first
NVRC_IMAGE_BGRA	1	Non-planar; 1 byte per channel; blue channel first
NVRC_IMAGE_NV12	2	Semi-planar YUV; 1 byte per channel; all luma samples first; chroma channels interleaved; each chroma sample corresponds to a 2x2 region of pixels.

nvrcImageType

A value that specifies whether the image is stored in a CPU buffer, or as either a CUDA or NVMEDIA image on GPU.

Enumerator	Value	Description
NVRC_IMAGE_CPU	0
NVRC_IMAGE_CUDA	1
NVRC_IMAGE_NVMEDIA	2

nvrcInputType

The type of the input event.

Enumerator	Value	Description
NVRC_INPUT_NONE	0
NVRC_INPUT_USER_EVENT	1

nvrcProjectionType

A value that specifies how the image of a camera is projected.

Enumerator	Value	Description
NVRC_PROJECTION_PINHOLE	0	Pinhole camera
NVRC_PROJECTION_FTHETA	1	Fisheye camera

nvrcStatus

The error return value from most NVRC functions.

Enumerator	Value	Description
NVRC_SUCCESS	0
NVRC_ERROR_INVALID_ARGUMENT	-1
NVRC_ERROR_FATAL	-2
NVRC_ERROR_RESOURCE_EXHAUSTED	-3
NVRC_ERROR_TIMEOUT	-4
NVRC_ERROR_END_OF_STREAM	-5
NVRC_ERROR_INVALID_CAMERA	-6
NVRC_ERROR_INVALID_SEQUENCE	-7
NVRC_ERROR_COMMUNICATION_FAILURE	-8
NVRC_ERROR_FRAME_NOT_AVAILABLE	-9
NVRC_ERROR_DECODER_FAILURE	-10
NVRC_ERROR_NOT_IMPLEMENTED	-11
NVRC_ERROR_NOT_INITIALIZED	-12
NVRC_ERROR_INVALID_CONFIG	-13
NVRC_ERROR_VERSION_MISMATCH	-14
NVRC_ERROR_INTERNAL_ERROR	-100
NVRC_ERROR_UNEXPECTED_EXCEPTION	-101

nvrcStreamType

The type of stream that the latency API is dealing with.

Enumerator	Value	Description
NVRC_STREAM_DATA
NVRC_STREAM_VIDEO
NUM_NVRC_STREAM_TYPES		The number of stream types that exist in this enum

nvrcSynchronizationType

The frequency control for data messages.

Enumerator	Value	Description
NVRC_NO_SYNCHRONIZATION	0	Data is sent as soon as it can be sent.
NVRC_SYNCHRONIZE_WITH_VIDEO	1	Not yet supported.
NVRC_SYNCHRONIZE_WITH_DATA_GROUP	2	Not yet supported.

Overview

Data Structures

Structure	Description
nvrcConnectionInfo	Information about a client connection
nvrcConnectionList	A collection of client connections
nvrcServerCallbacks	All callbacks associated with a particular server

Macros

Macro	Value	Description
NVRC_MAX_CONNECTIONS	8	The maximum number of client connections in an nvrcConnectionList
NVRC_MAX_IDENTIFIER_LENGTH	50	The maximum length of the string identifying a connection

Typedefs

Typedef	Description
typedef struct nvrcConnectionHandle * nvrcConnectionHandle_t	The identifier for a client connection
typedef struct nvrcServer * nvrcServer_t	An opaque handle to nvrcServer

Functions

Function	Description
nvrcServerAddLatencyMeasurementCallback	Add a periodic callback with an nvrcLatencyData structure containing the incoming data for that period as its argument.
nvrcServerClearLatencyMeasurementCallback	Clear the callback for latency data on incoming data.
nvrcServerCreate	Create the nvrcServer.
nvrcServerDestroy	Destroy the nvrcServer.
nvrcServerGetConnectionStatus	Check the server streaming status.
nvrcServerGetVersion	Retrieve the version with which the library was compiled.
nvrcServerSendData	Send generic data to the specified client.
nvrcServerSendVideo	Send a video frame to the specified client.
nvrcServerStartLatencyMeasurement	Start latency measurements for client control data.
nvrcServerStopLatencyMeasurement	Stop latency measurements and get latency information.

Data Structures

nvrcConnectionInfo

Information about a client connection. The definition is at line 54 of the file connection.h.

nvrcConnectionList

A collection of client connections. The definition is at line 74 of the file connection.h.

nvrcServerCallbacks

All callbacks associated with a particular server. Unless otherwise specified, each callback must be defined. The definition is at line 46 of the file callbacks.h.

The following are defined:

context

Copy
Copied!

            
            void* nvrcServerCallbacks::context

Passed into each callback.

You can also initialize this to a buffer or datastructure that you would like to access in the function bodies of each callback.

onClientCommand

Copy
Copied!

            
            void(* nvrcServerCallbacks::onClientCommand) (void *context, nvrcConnection info, void *command)

Called on input from streaming client.

The command parameter is always an nvrcInputData data structure.

onConnection

Copy
Copied!

            
            void(* nvrcServerCallbacks::onConnection) (void *context, nvrcConnection info)

Called when a streaming client connects.

You may leave this undefined (null pointer).

onDisconnection

Copy
Copied!

            
            void(* nvrcServerCallbacks::onDisconnection) (void *context, nvrcConnection info)

Called when a streaming client disconnects.

You may leave this undefined (null pointer).

onFailure

Copy
Copied!

            
            void(* nvrcServerCallbacks::onFailure )(void *context, nvrcConnection info, nvrcStatus status, const char *errorString)

onPause

Copy
Copied!

            
            void(* nvrcServerCallbacks::onPause) (void *context)

Called to instruct the server to pause sending video/data via NVRC.

You may leave this undefined (null pointer).

onPolicyTrigger

Copy
Copied!

            
            void(* nvrcServerCallbacks::onPolicyTrigger) (void *context, void *policy)

Reserved for future use.

You may leave this undefined (null pointer).

onReconfigure

Copy
Copied!

            
            nvrcStatus(* nvrcServerCallbacks::onReconfigure) (void *context, nvrcServerConfig config)

Called to instruct the server to re-configure providers based on the client request.

If NVRC determines that streaming also needs to restart, this callback will be called after onPause and before onResume.

onResume

Copy
Copied!

            
            void(* nvrcServerCallbacks::onResume) (void *context)

Called to instruct the server to resume sending video/data via NVRC.

You may leave this undefined (null pointer).

onStart

Copy
Copied!

            
            nvrcStatus(* nvrcServerCallbacks::onStart) (void *context, nvrcServerConfig config)

Called to inform the server that streaming has started and the client has requested the given config.

The server may use this to start sensors, data providers, etc.

onStop

Copy
Copied!

            
            nvrcStatus(* nvrcServerCallbacks::onStop) (void *context)

Called to inform the server that streaming will stop.

The server may use this to stop sensors, data providers, etc.

validateConfig

Copy
Copied!

            
            nvrcStatus(* nvrcServerCallbacks::validateConfig )(void *context, nvrcServerConfig config)

Functions

nvrcServerAddLatencyMeasurementCallback

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerAddLatencyMeasurementCallback ( nvrcServer_t                   server,
                                                                                                                                 nvrcConnectionHandle_t         connection,
                                                                                                                                 nvrcLatencyMeasurementCallback callback,
                                                             nvrcTime_t                     period_us
                                                           )

Add a periodic callback that receives an nvrcLatencyData for the incoming data of that period as its argument.

Parameters

[in] server: The handle to the nvrcServer
[in] connection: The handle to the client connection
[in] callback: The pointer to the function to be called, with latency data as the argument, after the period ends
[in] period_us: The number of microseconds between callbacks

Returns

NVRC_ERROR_INVALID_ARGUMENT: The server or callback is invalid.
NVRC_ERROR_INVALID_SEQUENCE: The normal latency measurement has started.
NVRC_SUCCESS: Success

nvrcServerClearLatencyMeasurementCallback

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerClearLatencyMeasurementCallback ( nvrcServer_t           server,
                                                                   nvrcConnectionHandle_t connection
                                                                 )

Clear the callback for latency data on incoming data.

Parameters

[in] server: The handle to the nvrcServer [in] connection: The handle to the client connection

Returns

NVRC_ERROR_INVALID_ARGUMENT: Server or callback is invalid.
NVRC_ERROR_INVALID_SEQUENCE: Normal latency measurement has been started or callback is not active.
NVRC_SUCCESS: Success

nvrcServerCreate

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerCreate ( nvrcServer_t *       server,
                                          const char *         identifier,
                                          uint16_t             port,
                                          nvrcServerConfig     config,
                                          nvrcServerCallbacks  callbacks
                                        )

Create the nvrcServer_.

Parameters

[out] server: The handle to the nvrcServer
[in] identifier: The string to identify the server. This parameter is currently unused.
[in] port: The port for the first video channel. This parameter is currently ignored; the port is always 48010
[in] config: The available config for the server. This parameter should include all all available cameras in the platformConfig_ if applicable. The client will receive this config and use it to determine which new config to request.
[in] callbacks: The server callbacks that should be defined by the app.

Returns

NVRC_ERROR_INVALID_ARGUMENT: The server is nullptr or config is invalid.
NVRC_ERROR_UNEXPECTED_EXCEPTION: An unexpected exception occurred while creating the server.
NVRC_SUCCESS: Success

nvrcServerDestroy

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerDestroy ( nvrcServer_t  server )

Destroy the nvrcServer_.

Parameters

[in] server: The handle to the nvrcServer_

Returns

NVRC_ERROR_INVALID_ARGUMENT: The server is invalid.
NVRC_SUCCESS: Success

nvrcServerGetConnectionStatus

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerGetConnectionStatus ( nvrcServer_t           server,
                                                        nvrcConnectionList *  connections
)

Check the server streaming status.

Parameters

[in] server: The handle to the nvrcServer_
[out] connections: A list of connections with status

Returns

NVRC_ERROR_INVALID_ARGUMENT: The server is invalid.
NVRC_ERROR_END_OF_STREAM: The server has no connections.
NVRC_SUCCESS: Success

nvrcServerGetVersion

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerGetVersion ( nvrcVersion *  version )

Gets the version with which the library was compiled.

Parameters

[out] version: The major, minor, patch, and hash version

Returns

NVRC_ERROR_INVALID_ARGUMENT: The version is nullptr.
NVRC_SUCCESS: Success

nvrcServerSendData

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerSendData ( nvrcServer_t             server,
                                            nvrcConnectionHandle_t   connection,
                                            nvrcData                 data,
                                            nvrcSynchronizationType  sync
                                          )

Send generic data to the specified client.

Parameters

[in] server: The handle to the nvrcServer
[in] connection: The connection to the client
[in] data: The data to send
[in] sync: Specifies whether or not the data should be sent at the same frequency as something else. Only NVRC_NO_SYNCHRONIZATION is supported.

Returns

NVRC_ERROR_INVALID_ARGUMENT: The data.ptr is invalid.
NVRC_ERROR_INVALID_SEQUENCE: The server has yet to be initialized.
NVRC_SUCCESS: Success

nvrcServerSendVideo

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerSendVideo ( nvrcServer_t           server,
                                             nvrcConnectionHandle_t connection,
                                             nvrcVideoFrame         frame,
                                             uint32_t               stream_id,
                                             void *                 metadata,
                                             size_t                 metadata_size
)

Send a video frame to the specified client.

Parameters

[in] server: The handle to the nvrcServer_
[in] connection: The connection to the client
[in] frame: The video frame to send
[in] stream_id: This should be the same value as frame.stream_id:
[in] metadata: Generic data to send along with the frame
[in] metadata_size: The number of bytes to send from the beginning of the metadata buffer

Returns

NVRC_ERROR_INVALID_ARGUMENT: Server is invalid.
NVRC_ERROR_INVALID_SEQUENCE Server has yet to be initialized.
NVRC_SUCCESS: Success

nvrcServerStartLatencyMeasurement

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerStartLatencyMeasurement ( nvrcServer_t            server,
                                                           nvrcConnectionHandle_t  connection
                                                         )

Start latency measurements for client control data.

Parameters

[in] server: The handle to the nvrcServer_
[in] connection: The handle to the client connection

Returns

NVRC_ERROR_INVALID_ARGUMENT: The server is invalid.
NVRC_ERROR_INVALID_SEQUENCE: The measurement has already started or callback is active.
NVRC_SUCCESS: Success

nvrcServerStopLatencyMeasurement

Copy
Copied!

            
            NVRC_EXPORT nvrcStatus nvrcServerStopLatencyMeasurement ( nvrcServer_t  server,
nvrcConnectionHandle_t  connection,
nvrcLatencyData *  latency_data
)

Stop latency measurements and get latency information.

Parameters

[in] server: The handle to the nvrcServer_
[in] connection: The handle to the client connection
[out] latency_data: The summary of latency since the last call to nvrcServerStartLatencyMeasurement

Returns

NVRC_ERROR_INVALID_ARGUMENT: The server or latency_data is invalid.
NVRC_ERROR_INVALID_SEQUENCE: The measurement has already started or callback is active.
NVRC_SUCCESS: Success