This is a reference page for the server API used within the Isaac RC module in the Isaac SDK app.
Overview
Data Structures
Structure | Description |
nvrcCameraProperties | Various details associated with a camera |
nvrcData | A message containing arbitrary data |
nvrcDataHeader | The metadata for a message |
nvrcDataStreamParams | Reserved for future use. |
nvrcImageParams | Metadata for a video frame |
nvrcInputData | An input event sent over the data stream |
nvrcLatencyData | Summary of latency data collected during a measurement period (since the last latency measurement callback) |
nvrcMosaicViewport | Various details associated with one section of a mosaic image |
nvrcServerConfig | Platform and streaming-specific information about the server |
nvrcStreamingParams | Various details associated with multiple video streams or a mosaic stream, as well as a data stream |
nvrcUserInputEvent | The contents of the user input event |
nvrcVideoFrame | A frame of video containing metadata and a buffer for the data of the frame itself |
nvrcVideoFrameGroup | A collection of video frames |
nvrcVideoStreamParams | Various details associated with a video stream |
Macros
Macro | Value | Description |
NVRC_MAX_DATA_LABEL_LENGTH | 50 | The maximum length of the label in a data message. |
NVRC_MAX_STREAM_NAME_LENGTH | 50 | Maximum length of the name of a stream or viewport. |
NVRC_MAX_VIDEO_FRAME_GROUP_COUNT | 4 | The maximum number of frames in a frame group. |
NVRC_MAX_VIDEO_STREAMS | 4 | Maximum number of video streams in a nvrcStreamingParams. |
Typedefs
Typedef | Description |
void (* nvrcLatencyMeasurementCallback) | |
uint64_t nvrcTime_t | The timestamp for a video frame or data message |
Enumerations
Enum | Description |
nvrcConnectionStatus | The status of a client connection |
nvrcImageFormat | The organization of color channels in the frame buffer |
nvrcImageType | A value that specifies whether the image is stored in a CPU buffer, or as either a CUDA or NVMEDIA image on GPU |
nvrcInputType | The type of the input event |
nvrcProjectionType | A value that specifies how the image of a camera is projected |
nvrcStatus | The error return value from most NVRC functions |
nvrcStreamType | The type of stream that the latency API is dealing with |
nvrcSynchronizationType | The frequency control for data messages |
Data Structures
nvrcCameraProperties
Various details associated with a camera. Definition is at line 62 of file config.h
.
Data Fields
Data Type | Parameter | Description |
float | bwPoly[5] | Backward-polynomial coefficients for a fisheye camera |
float | opticalCenter[2] | The position of the optical center in pixels, laid out in the array as [x, y] |
nvrcProjectionType | projectionType | |
float | rotation[4] | The quaternion rotation of the camera relative to the forward direction of the vehicle, laid out in the array as [w, x, y, z] |
float | translation[3] | The location of the camera relative to the origin of the vehicle in meters, laid out in the array as [x, y, z] |
nvrcData
A message containing arbitrary data. The definition is at line 183 of file frame.h
.

Data Type | Parameter | Description |
nvrcDataHeader | header | The metadata of the message |
void * | ptr | A CPU buffer containing the data of the message |
nvrcDataHeader
The metadata for a message. The definition is at line 153 of file frame.h
.
Data Type | Parameter | Description |
uint32_t | group_id | Reserved for future use. |
char | label[NVRC_MAX_DATA_LABEL_LENGTH] | An arbitrary label. Can be used to differentiate between types of messages. |
size_t | size | The number of bytes owned by the message from the beginning of the pointer provided in the parent nvrcData. |
nvrcTime_t | timestamp_us | Epoc time in microseconds, typically just before the message is given to NVRC. This figure is used to measure latency and synchronize with video, so be deliberate about when this timestamp is recorded. |
uint32_t | video_stream_id | Reserved for future use. |
nvrcDataStreamParams
Reserved for future use. The definition is at line 126 of file config.h
.
nvrcImageParams
The metadata for a video frame. The definition is at line 80 of file frame.h
.
Data Type | Parameter | Description |
nvrcImageFormat | format | The format of the frame buffer |
uint32_t | height | The number of pixels down in the image |
uint32_t | pitch | The number of bytes between the start of consecutive rows of the image |
nvrcImageType | type | The type of the frame buffer |
uint32_t | width | The number of pixels across in the image |
nvrcInputData
An input event sent over the data stream. The definition is at line 74 of file input.h
.

Data Type | Parameter | Description |
union InputUnion | inputUnion | |
nvrcTime_t | timestamp_us | A timestamp set by the sender of the input event |
nvrcInputType | type | The type corresponding to the alternative to choose in the inputUnion |
nvrcLatencyData
The summary of latency data collected during a measurement period (since the last latency measurement callback).
Latencies are based on the timestamps present in the received packets, the system epoc time in microseconds, and some synchronization between the clocks of this node and the sender. If the packets are from some pre-recorded dataset, or if the timestamps are not formatted as epoc time in microseconds, this latency data will likely be nonsensical.
The definition is at line 65 of file latency.h
.
Data Type | Parameter | Description |
float | average_us | The average latency in microseconds of the packets received during the measurement period |
nvrcTime_t_ | max_us | The maximum latency in microseconds of the packets received during the measurement period |
nvrcTime_t_ | min_us | The minimum latency in microseconds of the packets received during the measurement period |
uint32_t | num_packets | The number of packets received by this node during the measurement period |
float | std_dev_us | The standard deviation of the latency in microseconds of the packets received during the measurement period |
nvrcTime_t_ | time_measured_us | Microseconds elapsed during the measurement period |
nvrcMosaicViewport
Various details associated with one section of a mosaic image.
The definition is at line 134 of the file config.h
.
Data Type | Parameter | Description |
uint32_t | height | The number of pixels down in the images produced |
char | name[NVRC_MAX_STREAM_NAME_LENGTH] | The name of the viewport (e.g. the name of the sensor that produces it) |
uint32_t | width | The number of pixels across in the images produced |
uint32_t | x | The number of pixels to shift the viewport right |
uint32_t | y | The number of pixels to shift the viewport down |
nvrcServerConfig
Platform and streaming-specific information about the server. The server uses this data to send a default or available config to the client, and the client responds with their desired config.
The definition is at line 210 of the file config.h
.

Data Type | Parameter | Description |
const char * | platformConfig | A JSON string describing the server platform config |
nvrcStreamingParams | streamingParams | An object describing the server streaming config |
nvrcStreamingParams
Details associated with multiple video streams or a mosaic stream, as well as a data stream.
The definition is at line 168 of the file config.h
.
Data Type | Parameter | Description |
nvrcDataStreamParams | dataStream | The data stream managed by the server |
bool | mosaic | Specifies whether the server is in mosaic mode: There should only be one videoStream specified, which streams the composition of all viewports into a mosaic, and each stream specifies which viewport to use. |
uint32_t | mosaicViewportCount | The number of items from the beginning of the mosaicViewports array that are valid. |
nvrcMosaicViewport | mosaicViewports[NVRC_MAX_VIDEO_STREAMS] | One or more mosaic viewports managed by the server |
uint32_t | videoStreamCount | The number of valid items from the beginning of the videoStreams array |
nvrcVideoStreamParams | videoStreams[NVRC_MAX_VIDEO_STREAMS] | One or more video streams managed by the server |
nvrcUserInputEvent
The content of a user input event.
The definition at line 58 of the file input.h
.
Data Type | Parameter | Description |
uint8_t * | data | The CPU buffer that contains the content of the user input event |
uint32_t | size | The number of bytes from the beginning of the data buffer that represent the content of the user input event |
nvrcVideoFrame
A frame of video containing some metadata and a buffer for the data of the frame itself.
The definition is at line 106 of the file frame.h
.

Data Type | Parameter | Description |
nvrcImageParams | params | Specifies how the data in the frame is organized. |
void * | ptr | A buffer containing the frame data. The location of the buffer should correspond to the type specified in params . |
uint32_t | stream_id | There may be multiple streams that can send data in parallel. Video sent on stream 0 will be received on stream 0. |
nvrcTime_t | timestamp_us | The epoc time in microseconds: typically just before the frame is given to NVRC. Because this value is used to measure latency and synchronize with data, ensure that this timestamp is recorded at the correct moment. |
nvrcVideoFrameGroup
A collection of video frames.
The definition is at line 137 of the file frame.h
.
Data Type | Parameter | Description |
uint32_t | count | The number of frames used in the frames array |
nvrcVideoFrame | frames[NVRC_MAX_VIDEO_FRAME_GROUP_COUNT] | The frames in the group |
nvrcVideoStreamParams
Various details associated with a video stream.
The definition is at line 97 of the file config.h
.

Data Type | Parameter | Description |
nvrcCameraProperties | cameraProperties | The properties of the camera producing the stream |
uint32_t | height | The number of pixels down in the images produced |
uint32_t | maxBitrate | The maximum bits per second of the stream |
char | name[NVRC_MAX_STREAM_NAME_LENGTH] | The name of the stream; for instance, the name of the sensor that produces it |
float | refreshRate | The frames per second of the stream |
uint32_t | width | The number of pixels across in the images produced |
Enumerations
nvrcConnectionStatus
The status of a client connection.
Enumerator | Value | Description |
NVRC_CLIENT_CONNECTED | 0 | |
NVRC_CLIENT_RECONNECTING | 1 | |
NVRC_CLIENT_DISCONNECTED | 2 |
nvrcImageFormat
The organization of color channels in the frame buffer.
Enumerator | Value | Description |
NVRC_IMAGE_RGBA | 0 | Non-planar; 1 byte per channel; red channel first |
NVRC_IMAGE_BGRA | 1 | Non-planar; 1 byte per channel; blue channel first |
NVRC_IMAGE_NV12 | 2 | Semi-planar YUV; 1 byte per channel; all luma samples first; chroma channels interleaved; each chroma sample corresponds to a 2x2 region of pixels. |
nvrcImageType
A value that specifies whether the image is stored in a CPU buffer, or as either a CUDA or NVMEDIA image on GPU.
Enumerator | Value | Description |
NVRC_IMAGE_CPU | 0 | |
NVRC_IMAGE_CUDA | 1 | |
NVRC_IMAGE_NVMEDIA | 2 |
nvrcInputType
The type of the input event.
Enumerator | Value | Description |
NVRC_INPUT_NONE | 0 | |
NVRC_INPUT_USER_EVENT | 1 |
nvrcProjectionType
A value that specifies how the image of a camera is projected.
Enumerator | Value | Description |
NVRC_PROJECTION_PINHOLE | 0 | Pinhole camera |
NVRC_PROJECTION_FTHETA | 1 | Fisheye camera |
nvrcStatus
The error return value from most NVRC functions.
Enumerator | Value | Description |
NVRC_SUCCESS | 0 | |
NVRC_ERROR_INVALID_ARGUMENT | -1 | |
NVRC_ERROR_FATAL | -2 | |
NVRC_ERROR_RESOURCE_EXHAUSTED | -3 | |
NVRC_ERROR_TIMEOUT | -4 | |
NVRC_ERROR_END_OF_STREAM | -5 | |
NVRC_ERROR_INVALID_CAMERA | -6 | |
NVRC_ERROR_INVALID_SEQUENCE | -7 | |
NVRC_ERROR_COMMUNICATION_FAILURE | -8 | |
NVRC_ERROR_FRAME_NOT_AVAILABLE | -9 | |
NVRC_ERROR_DECODER_FAILURE | -10 | |
NVRC_ERROR_NOT_IMPLEMENTED | -11 | |
NVRC_ERROR_NOT_INITIALIZED | -12 | |
NVRC_ERROR_INVALID_CONFIG | -13 | |
NVRC_ERROR_VERSION_MISMATCH | -14 | |
NVRC_ERROR_INTERNAL_ERROR | -100 | |
NVRC_ERROR_UNEXPECTED_EXCEPTION | -101 |
nvrcStreamType
The type of stream that the latency API is dealing with.
Enumerator | Value | Description |
NVRC_STREAM_DATA | ||
NVRC_STREAM_VIDEO | ||
NUM_NVRC_STREAM_TYPES | The number of stream types that exist in this enum |
nvrcSynchronizationType
The frequency control for data messages.
Enumerator | Value | Description |
NVRC_NO_SYNCHRONIZATION | 0 | Data is sent as soon as it can be sent. |
NVRC_SYNCHRONIZE_WITH_VIDEO | 1 | Not yet supported. |
NVRC_SYNCHRONIZE_WITH_DATA_GROUP | 2 | Not yet supported. |
Overview
Data Structures
Structure | Description |
nvrcConnectionInfo | Information about a client connection |
nvrcConnectionList | A collection of client connections |
nvrcServerCallbacks | All callbacks associated with a particular server |
Macros
Macro | Value | Description |
NVRC_MAX_CONNECTIONS | 8 | The maximum number of client connections in an nvrcConnectionList |
NVRC_MAX_IDENTIFIER_LENGTH | 50 | The maximum length of the string identifying a connection |
Typedefs
Typedef | Description |
typedef struct nvrcConnectionHandle * nvrcConnectionHandle_t | The identifier for a client connection |
typedef struct nvrcServer * nvrcServer_t | An opaque handle to nvrcServer |
Functions
Function | Description |
nvrcServerAddLatencyMeasurementCallback | Add a periodic callback with an nvrcLatencyData structure containing the incoming data for that period as its argument. |
nvrcServerClearLatencyMeasurementCallback | Clear the callback for latency data on incoming data. |
nvrcServerCreate | Create the nvrcServer. |
nvrcServerDestroy | Destroy the nvrcServer. |
nvrcServerGetConnectionStatus | Check the server streaming status. |
nvrcServerGetVersion | Retrieve the version with which the library was compiled. |
nvrcServerSendData | Send generic data to the specified client. |
nvrcServerSendVideo | Send a video frame to the specified client. |
nvrcServerStartLatencyMeasurement | Start latency measurements for client control data. |
nvrcServerStopLatencyMeasurement | Stop latency measurements and get latency information. |
Data Structures
nvrcConnectionInfo
Information about a client connection. The definition is at line 54 of the file
connection.h
.
nvrcConnectionList
A collection of client connections. The definition is at line 74 of the file connection.h
.
nvrcServerCallbacks
All callbacks associated with a particular server. Unless otherwise specified, each callback must
be defined. The definition is at line 46 of the file callbacks.h
.
The following are defined:
context
void* nvrcServerCallbacks::context
Passed into each callback.
You can also initialize this to a buffer or datastructure that you would like to access in the function bodies of each callback.
onClientCommand
void(* nvrcServerCallbacks::onClientCommand) (void *context, nvrcConnection info, void *command)
Called on input from streaming client.
The command parameter is always an nvrcInputData data structure.
onConnection
void(* nvrcServerCallbacks::onConnection) (void *context, nvrcConnection info)
Called when a streaming client connects.
You may leave this undefined (null pointer).
onDisconnection
void(* nvrcServerCallbacks::onDisconnection) (void *context, nvrcConnection info)
Called when a streaming client disconnects.
You may leave this undefined (null pointer).
onFailure
void(* nvrcServerCallbacks::onFailure )(void *context, nvrcConnection info, nvrcStatus status, const char *errorString)
onPause
void(* nvrcServerCallbacks::onPause) (void *context)
Called to instruct the server to pause sending video/data via NVRC.
You may leave this undefined (null pointer).
onPolicyTrigger
void(* nvrcServerCallbacks::onPolicyTrigger) (void *context, void *policy)
Reserved for future use.
You may leave this undefined (null pointer).
onReconfigure
nvrcStatus(* nvrcServerCallbacks::onReconfigure) (void *context, nvrcServerConfig config)
Called to instruct the server to re-configure providers based on the client request.
If NVRC determines that streaming also needs to restart, this callback will be called after
onPause
and before onResume
.
onResume
void(* nvrcServerCallbacks::onResume) (void *context)
Called to instruct the server to resume sending video/data via NVRC.
You may leave this undefined (null pointer).
onStart
nvrcStatus(* nvrcServerCallbacks::onStart) (void *context, nvrcServerConfig config)
Called to inform the server that streaming has started and the client has requested the given config.
The server may use this to start sensors, data providers, etc.
onStop
nvrcStatus(* nvrcServerCallbacks::onStop) (void *context)
Called to inform the server that streaming will stop.
The server may use this to stop sensors, data providers, etc.
validateConfig
nvrcStatus(* nvrcServerCallbacks::validateConfig )(void *context, nvrcServerConfig config)
Functions
nvrcServerAddLatencyMeasurementCallback
NVRC_EXPORT nvrcStatus nvrcServerAddLatencyMeasurementCallback ( nvrcServer_t server,
nvrcConnectionHandle_t connection,
nvrcLatencyMeasurementCallback callback,
nvrcTime_t period_us
)
Add a periodic callback that receives an nvrcLatencyData for the incoming data of that period as its argument.
Parameters
- [in]
server
: The handle to the nvrcServer - [in]
connection
: The handle to the client connection - [in]
callback
: The pointer to the function to be called, with latency data as the argument, after the period ends - [in]
period_us
: The number of microseconds between callbacks
Returns
NVRC_ERROR_INVALID_ARGUMENT
: The server or callback is invalid.NVRC_ERROR_INVALID_SEQUENCE
: The normal latency measurement has started.NVRC_SUCCESS
: Success
nvrcServerClearLatencyMeasurementCallback
NVRC_EXPORT nvrcStatus nvrcServerClearLatencyMeasurementCallback ( nvrcServer_t server,
nvrcConnectionHandle_t connection
)
Clear the callback for latency data on incoming data.
Parameters
[in] server
: The handle to the nvrcServer
[in] connection
: The handle to the client connection
Returns
NVRC_ERROR_INVALID_ARGUMENT
: Server or callback is invalid.NVRC_ERROR_INVALID_SEQUENCE
: Normal latency measurement has been started or callback is not active.NVRC_SUCCESS
: Success
nvrcServerCreate
NVRC_EXPORT nvrcStatus nvrcServerCreate ( nvrcServer_t * server,
const char * identifier,
uint16_t port,
nvrcServerConfig config,
nvrcServerCallbacks callbacks
)
Create the nvrcServer_.
Parameters
- [out]
server
: The handle to the nvrcServer - [in]
identifier
: The string to identify the server. This parameter is currently unused. - [in]
port
: The port for the first video channel. This parameter is currently ignored; the port is always 48010 - [in]
config
: The available config for the server. This parameter should include all all available cameras in the platformConfig_ if applicable. The client will receive this config and use it to determine which new config to request. - [in]
callbacks
: The server callbacks that should be defined by the app.
Returns
NVRC_ERROR_INVALID_ARGUMENT
: The server is nullptr or config is invalid.NVRC_ERROR_UNEXPECTED_EXCEPTION
: An unexpected exception occurred while creating the server.NVRC_SUCCESS
: Success
nvrcServerDestroy
NVRC_EXPORT nvrcStatus nvrcServerDestroy ( nvrcServer_t server )
Destroy the nvrcServer_.
Parameters
- [in]
server
: The handle to the nvrcServer_
Returns
NVRC_ERROR_INVALID_ARGUMENT
: The server is invalid.NVRC_SUCCESS
: Success
nvrcServerGetConnectionStatus
NVRC_EXPORT nvrcStatus nvrcServerGetConnectionStatus ( nvrcServer_t server,
nvrcConnectionList * connections
)
Check the server streaming status.
Parameters
- [in]
server
: The handle to the nvrcServer_ - [out]
connections
: A list of connections with status
Returns
NVRC_ERROR_INVALID_ARGUMENT
: The server is invalid.NVRC_ERROR_END_OF_STREAM
: The server has no connections.NVRC_SUCCESS
: Success
nvrcServerGetVersion
NVRC_EXPORT nvrcStatus nvrcServerGetVersion ( nvrcVersion * version )
Gets the version with which the library was compiled.
Parameters
- [out]
version
: The major, minor, patch, and hash version
Returns
NVRC_ERROR_INVALID_ARGUMENT
: The version is nullptr.NVRC_SUCCESS
: Success
nvrcServerSendData
NVRC_EXPORT nvrcStatus nvrcServerSendData ( nvrcServer_t server,
nvrcConnectionHandle_t connection,
nvrcData data,
nvrcSynchronizationType sync
)
Send generic data to the specified client.
Parameters
- [in]
server
: The handle to the nvrcServer - [in]
connection
: The connection to the client - [in]
data
: The data to send - [in]
sync
: Specifies whether or not the data should be sent at the same frequency as something else. OnlyNVRC_NO_SYNCHRONIZATION
is supported.
Returns
NVRC_ERROR_INVALID_ARGUMENT
: Thedata.ptr
is invalid.NVRC_ERROR_INVALID_SEQUENCE
: The server has yet to be initialized.NVRC_SUCCESS
: Success
nvrcServerSendVideo
NVRC_EXPORT nvrcStatus nvrcServerSendVideo ( nvrcServer_t server,
nvrcConnectionHandle_t connection,
nvrcVideoFrame frame,
uint32_t stream_id,
void * metadata,
size_t metadata_size
)
Send a video frame to the specified client.
Parameters
- [in]
server
: The handle to the nvrcServer_ - [in]
connection
: The connection to the client - [in]
frame
: The video frame to send - [in]
stream_id
: This should be the same value asframe.stream_id
: - [in]
metadata
: Generic data to send along with the frame - [in]
metadata_size
: The number of bytes to send from the beginning of the metadata buffer
Returns
NVRC_ERROR_INVALID_ARGUMENT
: Server is invalid.NVRC_ERROR_INVALID_SEQUENCE
Server has yet to be initialized.NVRC_SUCCESS
: Success
nvrcServerStartLatencyMeasurement
NVRC_EXPORT nvrcStatus nvrcServerStartLatencyMeasurement ( nvrcServer_t server,
nvrcConnectionHandle_t connection
)
Start latency measurements for client control data.
Parameters
- [in]
server
: The handle to the nvrcServer_ - [in]
connection
: The handle to the client connection
Returns
NVRC_ERROR_INVALID_ARGUMENT
: The server is invalid.NVRC_ERROR_INVALID_SEQUENCE
: The measurement has already started or callback is active.NVRC_SUCCESS
: Success
nvrcServerStopLatencyMeasurement
NVRC_EXPORT nvrcStatus nvrcServerStopLatencyMeasurement ( nvrcServer_t server,
nvrcConnectionHandle_t connection,
nvrcLatencyData * latency_data
)
Stop latency measurements and get latency information.
Parameters
- [in]
server
: The handle to the nvrcServer_ - [in]
connection
: The handle to the client connection - [out]
latency_data
: The summary of latency since the last call to nvrcServerStartLatencyMeasurement
Returns
NVRC_ERROR_INVALID_ARGUMENT
: The server orlatency_data
is invalid.NVRC_ERROR_INVALID_SEQUENCE
: The measurement has already started or callback is active.NVRC_SUCCESS
: Success