NVRC Server API

This is a reference page for the server API used within the Isaac RC module in the Isaac SDK app.

Overview

Data Structures

Structure Description
nvrcCameraProperties Various details associated with a camera
nvrcData A message containing arbitrary data
nvrcDataHeader The metadata for a message
nvrcDataStreamParams Reserved for future use.
nvrcImageParams Metadata for a video frame
nvrcInputData An input event sent over the data stream
nvrcLatencyData Summary of latency data collected during a measurement period (since the last latency measurement callback)
nvrcMosaicViewport Various details associated with one section of a mosaic image
nvrcServerConfig Platform and streaming-specific information about the server
nvrcStreamingParams Various details associated with multiple video streams or a mosaic stream, as well as a data stream
nvrcUserInputEvent The contents of the user input event
nvrcVideoFrame A frame of video containing metadata and a buffer for the data of the frame itself
nvrcVideoFrameGroup A collection of video frames
nvrcVideoStreamParams Various details associated with a video stream

Macros

Macro Value Description
NVRC_MAX_DATA_LABEL_LENGTH 50 The maximum length of the label in a data message.
NVRC_MAX_STREAM_NAME_LENGTH 50 Maximum length of the name of a stream or viewport.
NVRC_MAX_VIDEO_FRAME_GROUP_COUNT 4 The maximum number of frames in a frame group.
NVRC_MAX_VIDEO_STREAMS 4 Maximum number of video streams in a nvrcStreamingParams.

Typedefs

Typedef Description
void (* nvrcLatencyMeasurementCallback)
uint64_t nvrcTime_t The timestamp for a video frame or data message

Enumerations

Enum Description
nvrcConnectionStatus The status of a client connection
nvrcImageFormat The organization of color channels in the frame buffer
nvrcImageType A value that specifies whether the image is stored in a CPU buffer, or as either a CUDA or NVMEDIA image on GPU
nvrcInputType The type of the input event
nvrcProjectionType A value that specifies how the image of a camera is projected
nvrcStatus The error return value from most NVRC functions
nvrcStreamType The type of stream that the latency API is dealing with
nvrcSynchronizationType The frequency control for data messages

Data Structures

nvrcCameraProperties

Various details associated with a camera. Definition is at line 62 of file config.h.

Data Fields

Data Type Parameter Description
float bwPoly[5] Backward-polynomial coefficients for a fisheye camera
float opticalCenter[2] The position of the optical center in pixels, laid out in the array as [x, y]
nvrcProjectionType projectionType  
float rotation[4] The quaternion rotation of the camera relative to the forward direction of the vehicle, laid out in the array as [w, x, y, z]
float translation[3] The location of the camera relative to the origin of the vehicle in meters, laid out in the array as [x, y, z]

nvrcData

A message containing arbitrary data. The definition is at line 183 of file frame.h.

structnvrcData__coll__graph1.jpg

Data Type Parameter Description
nvrcDataHeader header The metadata of the message
void * ptr A CPU buffer containing the data of the message

nvrcDataHeader

The metadata for a message. The definition is at line 153 of file frame.h.

Data Type Parameter Description
uint32_t group_id Reserved for future use.
char label[NVRC_MAX_DATA_LABEL_LENGTH] An arbitrary label. Can be used to differentiate between types of messages.
size_t size The number of bytes owned by the message from the beginning of the pointer provided in the parent nvrcData.
nvrcTime_t timestamp_us Epoc time in microseconds, typically just before the message is given to NVRC. This figure is used to measure latency and synchronize with video, so be deliberate about when this timestamp is recorded.
uint32_t video_stream_id Reserved for future use.

nvrcDataStreamParams

Reserved for future use. The definition is at line 126 of file config.h.

nvrcImageParams

The metadata for a video frame. The definition is at line 80 of file frame.h.

Data Type Parameter Description
nvrcImageFormat format The format of the frame buffer
uint32_t height The number of pixels down in the image
uint32_t pitch The number of bytes between the start of consecutive rows of the image
nvrcImageType type The type of the frame buffer
uint32_t width The number of pixels across in the image

nvrcInputData

An input event sent over the data stream. The definition is at line 74 of file input.h.

structnvrcInputData__coll__graph1.jpg

Data Type Parameter Description
union InputUnion inputUnion  
nvrcTime_t timestamp_us A timestamp set by the sender of the input event
nvrcInputType type The type corresponding to the alternative to choose in the inputUnion

nvrcLatencyData

The summary of latency data collected during a measurement period (since the last latency measurement callback).

Latencies are based on the timestamps present in the received packets, the system epoc time in microseconds, and some synchronization between the clocks of this node and the sender. If the packets are from some pre-recorded dataset, or if the timestamps are not formatted as epoc time in microseconds, this latency data will likely be nonsensical.

The definition is at line 65 of file latency.h.

Data Type Parameter Description
float average_us The average latency in microseconds of the packets received during the measurement period
nvrcTime_t_ max_us The maximum latency in microseconds of the packets received during the measurement period
nvrcTime_t_ min_us The minimum latency in microseconds of the packets received during the measurement period
uint32_t num_packets The number of packets received by this node during the measurement period
float std_dev_us The standard deviation of the latency in microseconds of the packets received during the measurement period
nvrcTime_t_ time_measured_us Microseconds elapsed during the measurement period

nvrcMosaicViewport

Various details associated with one section of a mosaic image.

The definition is at line 134 of the file config.h.

Data Type Parameter Description
uint32_t height The number of pixels down in the images produced
char name[NVRC_MAX_STREAM_NAME_LENGTH] The name of the viewport (e.g. the name of the sensor that produces it)
uint32_t width The number of pixels across in the images produced
uint32_t x The number of pixels to shift the viewport right
uint32_t y The number of pixels to shift the viewport down

nvrcServerConfig

Platform and streaming-specific information about the server. The server uses this data to send a default or available config to the client, and the client responds with their desired config.

The definition is at line 210 of the file config.h.

structnvrcServerConfig__coll__graph1.jpg

Data Type Parameter Description
const char * platformConfig A JSON string describing the server platform config
nvrcStreamingParams streamingParams An object describing the server streaming config

nvrcStreamingParams

Details associated with multiple video streams or a mosaic stream, as well as a data stream.

The definition is at line 168 of the file config.h.

Data Type Parameter Description
nvrcDataStreamParams dataStream The data stream managed by the server
bool mosaic Specifies whether the server is in mosaic mode: There should only be one videoStream specified, which streams the composition of all viewports into a mosaic, and each stream specifies which viewport to use.
uint32_t mosaicViewportCount The number of items from the beginning of the mosaicViewports array that are valid.
nvrcMosaicViewport mosaicViewports[NVRC_MAX_VIDEO_STREAMS] One or more mosaic viewports managed by the server
uint32_t videoStreamCount The number of valid items from the beginning of the videoStreams array
nvrcVideoStreamParams videoStreams[NVRC_MAX_VIDEO_STREAMS] One or more video streams managed by the server

nvrcUserInputEvent

The content of a user input event.

The definition at line 58 of the file input.h.

Data Type Parameter Description
uint8_t * data The CPU buffer that contains the content of the user input event
uint32_t size The number of bytes from the beginning of the data buffer that represent the content of the user input event

nvrcVideoFrame

A frame of video containing some metadata and a buffer for the data of the frame itself.

The definition is at line 106 of the file frame.h.

structnvrcVideoFrame__coll__graph1.jpg

Data Type Parameter Description
nvrcImageParams params Specifies how the data in the frame is organized.
void * ptr A buffer containing the frame data. The location of the buffer should correspond to the type specified in params.
uint32_t stream_id There may be multiple streams that can send data in parallel. Video sent on stream 0 will be received on stream 0.
nvrcTime_t timestamp_us The epoc time in microseconds: typically just before the frame is given to NVRC. Because this value is used to measure latency and synchronize with data, ensure that this timestamp is recorded at the correct moment.

nvrcVideoFrameGroup

A collection of video frames.

The definition is at line 137 of the file frame.h.

Data Type Parameter Description
uint32_t count The number of frames used in the frames array
nvrcVideoFrame frames[NVRC_MAX_VIDEO_FRAME_GROUP_COUNT] The frames in the group

nvrcVideoStreamParams

Various details associated with a video stream.

The definition is at line 97 of the file config.h.

structnvrcVideoStreamParams__coll__graph1.jpg

Data Type Parameter Description
nvrcCameraProperties cameraProperties The properties of the camera producing the stream
uint32_t height The number of pixels down in the images produced
uint32_t maxBitrate The maximum bits per second of the stream
char name[NVRC_MAX_STREAM_NAME_LENGTH] The name of the stream; for instance, the name of the sensor that produces it
float refreshRate The frames per second of the stream
uint32_t width The number of pixels across in the images produced

Enumerations

nvrcConnectionStatus

The status of a client connection.

Enumerator Value Description
NVRC_CLIENT_CONNECTED 0
NVRC_CLIENT_RECONNECTING 1
NVRC_CLIENT_DISCONNECTED 2

nvrcImageFormat

The organization of color channels in the frame buffer.

Enumerator Value Description
NVRC_IMAGE_RGBA 0 Non-planar; 1 byte per channel; red channel first
NVRC_IMAGE_BGRA 1 Non-planar; 1 byte per channel; blue channel first
NVRC_IMAGE_NV12 2 Semi-planar YUV; 1 byte per channel; all luma samples first; chroma channels interleaved; each chroma sample corresponds to a 2x2 region of pixels.

nvrcImageType

A value that specifies whether the image is stored in a CPU buffer, or as either a CUDA or NVMEDIA image on GPU.

Enumerator Value Description
NVRC_IMAGE_CPU 0
NVRC_IMAGE_CUDA 1
NVRC_IMAGE_NVMEDIA 2

nvrcInputType

The type of the input event.

Enumerator Value Description
NVRC_INPUT_NONE 0
NVRC_INPUT_USER_EVENT 1

nvrcProjectionType

A value that specifies how the image of a camera is projected.

Enumerator Value Description
NVRC_PROJECTION_PINHOLE 0 Pinhole camera
NVRC_PROJECTION_FTHETA 1 Fisheye camera

nvrcStatus

The error return value from most NVRC functions.

Enumerator Value Description
NVRC_SUCCESS 0
NVRC_ERROR_INVALID_ARGUMENT -1
NVRC_ERROR_FATAL -2
NVRC_ERROR_RESOURCE_EXHAUSTED -3
NVRC_ERROR_TIMEOUT -4
NVRC_ERROR_END_OF_STREAM -5
NVRC_ERROR_INVALID_CAMERA -6
NVRC_ERROR_INVALID_SEQUENCE -7
NVRC_ERROR_COMMUNICATION_FAILURE -8
NVRC_ERROR_FRAME_NOT_AVAILABLE -9
NVRC_ERROR_DECODER_FAILURE -10
NVRC_ERROR_NOT_IMPLEMENTED -11
NVRC_ERROR_NOT_INITIALIZED -12
NVRC_ERROR_INVALID_CONFIG -13
NVRC_ERROR_VERSION_MISMATCH -14
NVRC_ERROR_INTERNAL_ERROR -100
NVRC_ERROR_UNEXPECTED_EXCEPTION -101

nvrcStreamType

The type of stream that the latency API is dealing with.

Enumerator Value Description
NVRC_STREAM_DATA
NVRC_STREAM_VIDEO
NUM_NVRC_STREAM_TYPES
The number of stream types that exist in this enum

nvrcSynchronizationType

The frequency control for data messages.

Enumerator Value Description
NVRC_NO_SYNCHRONIZATION 0 Data is sent as soon as it can be sent.
NVRC_SYNCHRONIZE_WITH_VIDEO 1 Not yet supported.
NVRC_SYNCHRONIZE_WITH_DATA_GROUP 2 Not yet supported.

Overview

Data Structures

Structure Description
nvrcConnectionInfo Information about a client connection
nvrcConnectionList A collection of client connections
nvrcServerCallbacks All callbacks associated with a particular server

Macros

Macro Value Description
NVRC_MAX_CONNECTIONS 8 The maximum number of client connections in an nvrcConnectionList
NVRC_MAX_IDENTIFIER_LENGTH 50 The maximum length of the string identifying a connection

Typedefs

Typedef Description
typedef struct nvrcConnectionHandle * nvrcConnectionHandle_t The identifier for a client connection
typedef struct nvrcServer * nvrcServer_t An opaque handle to nvrcServer

Functions

Function Description
nvrcServerAddLatencyMeasurementCallback Add a periodic callback with an nvrcLatencyData structure containing the incoming data for that period as its argument.
nvrcServerClearLatencyMeasurementCallback Clear the callback for latency data on incoming data.
nvrcServerCreate Create the nvrcServer.
nvrcServerDestroy Destroy the nvrcServer.
nvrcServerGetConnectionStatus Check the server streaming status.
nvrcServerGetVersion Retrieve the version with which the library was compiled.
nvrcServerSendData Send generic data to the specified client.
nvrcServerSendVideo Send a video frame to the specified client.
nvrcServerStartLatencyMeasurement Start latency measurements for client control data.
nvrcServerStopLatencyMeasurement Stop latency measurements and get latency information.

Data Structures

nvrcConnectionInfo

Information about a client connection. The definition is at line 54 of the file connection.h.

nvrcConnectionList

A collection of client connections. The definition is at line 74 of the file connection.h.

nvrcServerCallbacks

All callbacks associated with a particular server. Unless otherwise specified, each callback must be defined. The definition is at line 46 of the file callbacks.h.

The following are defined:

context

Copy
Copied!
            

void* nvrcServerCallbacks::context

Passed into each callback.

You can also initialize this to a buffer or datastructure that you would like to access in the function bodies of each callback.

onClientCommand

Copy
Copied!
            

void(* nvrcServerCallbacks::onClientCommand) (void *context, nvrcConnection info, void *command)

Called on input from streaming client.

The command parameter is always an nvrcInputData data structure.

onConnection

Copy
Copied!
            

void(* nvrcServerCallbacks::onConnection) (void *context, nvrcConnection info)

Called when a streaming client connects.

You may leave this undefined (null pointer).

onDisconnection

Copy
Copied!
            

void(* nvrcServerCallbacks::onDisconnection) (void *context, nvrcConnection info)

Called when a streaming client disconnects.

You may leave this undefined (null pointer).

onFailure

Copy
Copied!
            

void(* nvrcServerCallbacks::onFailure )(void *context, nvrcConnection info, nvrcStatus status, const char *errorString)

onPause

Copy
Copied!
            

void(* nvrcServerCallbacks::onPause) (void *context)

Called to instruct the server to pause sending video/data via NVRC.

You may leave this undefined (null pointer).

onPolicyTrigger

Copy
Copied!
            

void(* nvrcServerCallbacks::onPolicyTrigger) (void *context, void *policy)

Reserved for future use.

You may leave this undefined (null pointer).

onReconfigure

Copy
Copied!
            

nvrcStatus(* nvrcServerCallbacks::onReconfigure) (void *context, nvrcServerConfig config)

Called to instruct the server to re-configure providers based on the client request.

If NVRC determines that streaming also needs to restart, this callback will be called after onPause and before onResume.

onResume

Copy
Copied!
            

void(* nvrcServerCallbacks::onResume) (void *context)

Called to instruct the server to resume sending video/data via NVRC.

You may leave this undefined (null pointer).

onStart

Copy
Copied!
            

nvrcStatus(* nvrcServerCallbacks::onStart) (void *context, nvrcServerConfig config)

Called to inform the server that streaming has started and the client has requested the given config.

The server may use this to start sensors, data providers, etc.

onStop

Copy
Copied!
            

nvrcStatus(* nvrcServerCallbacks::onStop) (void *context)

Called to inform the server that streaming will stop.

The server may use this to stop sensors, data providers, etc.

validateConfig

Copy
Copied!
            

nvrcStatus(* nvrcServerCallbacks::validateConfig )(void *context, nvrcServerConfig config)

Functions

nvrcServerAddLatencyMeasurementCallback

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerAddLatencyMeasurementCallback ( nvrcServer_t server, nvrcConnectionHandle_t connection, nvrcLatencyMeasurementCallback callback, nvrcTime_t period_us )

Add a periodic callback that receives an nvrcLatencyData for the incoming data of that period as its argument.

Parameters

  • [in] server: The handle to the nvrcServer
  • [in] connection: The handle to the client connection
  • [in] callback: The pointer to the function to be called, with latency data as the argument, after the period ends
  • [in] period_us: The number of microseconds between callbacks

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The server or callback is invalid.
  • NVRC_ERROR_INVALID_SEQUENCE: The normal latency measurement has started.
  • NVRC_SUCCESS: Success

nvrcServerClearLatencyMeasurementCallback

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerClearLatencyMeasurementCallback ( nvrcServer_t server, nvrcConnectionHandle_t connection )

Clear the callback for latency data on incoming data.

Parameters

[in] server: The handle to the nvrcServer [in] connection: The handle to the client connection

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: Server or callback is invalid.
  • NVRC_ERROR_INVALID_SEQUENCE: Normal latency measurement has been started or callback is not active.
  • NVRC_SUCCESS: Success

nvrcServerCreate

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerCreate ( nvrcServer_t * server, const char * identifier, uint16_t port, nvrcServerConfig config, nvrcServerCallbacks callbacks )

Create the nvrcServer_.

Parameters

  • [out] server: The handle to the nvrcServer
  • [in] identifier: The string to identify the server. This parameter is currently unused.
  • [in] port: The port for the first video channel. This parameter is currently ignored; the port is always 48010
  • [in] config: The available config for the server. This parameter should include all all available cameras in the platformConfig_ if applicable. The client will receive this config and use it to determine which new config to request.
  • [in] callbacks: The server callbacks that should be defined by the app.

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The server is nullptr or config is invalid.
  • NVRC_ERROR_UNEXPECTED_EXCEPTION: An unexpected exception occurred while creating the server.
  • NVRC_SUCCESS: Success

nvrcServerDestroy

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerDestroy ( nvrcServer_t server )

Destroy the nvrcServer_.

Parameters

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The server is invalid.
  • NVRC_SUCCESS: Success

nvrcServerGetConnectionStatus

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerGetConnectionStatus ( nvrcServer_t server, nvrcConnectionList * connections )

Check the server streaming status.

Parameters

  • [in] server: The handle to the nvrcServer_
  • [out] connections: A list of connections with status

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The server is invalid.
  • NVRC_ERROR_END_OF_STREAM: The server has no connections.
  • NVRC_SUCCESS: Success

nvrcServerGetVersion

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerGetVersion ( nvrcVersion * version )

Gets the version with which the library was compiled.

Parameters

  • [out] version: The major, minor, patch, and hash version

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The version is nullptr.
  • NVRC_SUCCESS: Success

nvrcServerSendData

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerSendData ( nvrcServer_t server, nvrcConnectionHandle_t connection, nvrcData data, nvrcSynchronizationType sync )

Send generic data to the specified client.

Parameters

  • [in] server: The handle to the nvrcServer
  • [in] connection: The connection to the client
  • [in] data: The data to send
  • [in] sync: Specifies whether or not the data should be sent at the same frequency as something else. Only NVRC_NO_SYNCHRONIZATION is supported.

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The data.ptr is invalid.
  • NVRC_ERROR_INVALID_SEQUENCE: The server has yet to be initialized.
  • NVRC_SUCCESS: Success

nvrcServerSendVideo

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerSendVideo ( nvrcServer_t server, nvrcConnectionHandle_t connection, nvrcVideoFrame frame, uint32_t stream_id, void * metadata, size_t metadata_size )

Send a video frame to the specified client.

Parameters

  • [in] server: The handle to the nvrcServer_
  • [in] connection: The connection to the client
  • [in] frame: The video frame to send
  • [in] stream_id: This should be the same value as frame.stream_id:
  • [in] metadata: Generic data to send along with the frame
  • [in] metadata_size: The number of bytes to send from the beginning of the metadata buffer

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: Server is invalid.
  • NVRC_ERROR_INVALID_SEQUENCE Server has yet to be initialized.
  • NVRC_SUCCESS: Success

nvrcServerStartLatencyMeasurement

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerStartLatencyMeasurement ( nvrcServer_t server, nvrcConnectionHandle_t connection )

Start latency measurements for client control data.

Parameters

  • [in] server: The handle to the nvrcServer_
  • [in] connection: The handle to the client connection

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The server is invalid.
  • NVRC_ERROR_INVALID_SEQUENCE: The measurement has already started or callback is active.
  • NVRC_SUCCESS: Success

nvrcServerStopLatencyMeasurement

Copy
Copied!
            

NVRC_EXPORT nvrcStatus nvrcServerStopLatencyMeasurement ( nvrcServer_t server, nvrcConnectionHandle_t connection, nvrcLatencyData * latency_data )

Stop latency measurements and get latency information.

Parameters

  • [in] server: The handle to the nvrcServer_
  • [in] connection: The handle to the client connection
  • [out] latency_data: The summary of latency since the last call to nvrcServerStartLatencyMeasurement

Returns

  • NVRC_ERROR_INVALID_ARGUMENT: The server or latency_data is invalid.
  • NVRC_ERROR_INVALID_SEQUENCE: The measurement has already started or callback is active.
  • NVRC_SUCCESS: Success

© Copyright 2018-2020, NVIDIA Corporation. Last updated on Oct 30, 2023.