DriveWorks SDK Reference
4.0.0 Release
For Test and Development only

Camera Sensor

Detailed Description

Defines the camera module, which provides access to a virtual camera reading reading the data from a video file or a real camera connected to different sockets.

Unless explicitly specified, all errors returned by DW APIs are non recoverable and the user application should transition to fail safe mode. In addition, any error code not described in this documentation should be consider as fatal and the user application should also transition to fail safe mode.

Data Structures

struct  dwCameraProperties
 Camera Properties. More...
 

Typedefs

typedef struct dwCameraFrame * dwCameraFrameHandle_t
 Handle to captured frame. More...
 
typedef void NvMediaIPPManager
 

Enumerations

enum  dwCameraExposureControl {
  DW_CAMERA_EXPOSURE_NONE,
  DW_CAMERA_EXPOSURE_UNKNOWN,
  DW_CAMERA_EXPOSURE_AE,
  DW_CAMERA_EXPOSURE_BAE,
  DW_CAMERA_EXPOSURE_CUSTOM
}
 Enum of exposure control types. More...
 
enum  dwCameraFOV {
  DW_CAMERA_FOV_UNKNOWN = 0,
  DW_CAMERA_FOV_30 = 30,
  DW_CAMERA_FOV_48 = 48,
  DW_CAMERA_FOV_50 = 50,
  DW_CAMERA_FOV_55 = 55,
  DW_CAMERA_FOV_60 = 60,
  DW_CAMERA_FOV_65 = 65,
  DW_CAMERA_FOV_90 = 90,
  DW_CAMERA_FOV_100 = 100,
  DW_CAMERA_FOV_110 = 110,
  DW_CAMERA_FOV_120 = 120,
  DW_CAMERA_FOV_185 = 185
}
 Enum of available FOV in degrees for camera lenses. More...
 
enum  dwCameraOutputType {
  DW_CAMERA_OUTPUT_NATIVE_PROCESSED = 1 << 0,
  DW_CAMERA_OUTPUT_NATIVE_RAW = 1 << 1,
  DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8 = 1 << 2,
  DW_CAMERA_OUTPUT_CUDA_YUV420_UINT8_PLANAR = 1 << 3,
  DW_CAMERA_OUTPUT_CUDA_RAW_UINT16 = 1 << 4,
  DW_CAMERA_OUTPUT_CUDA_YUV420_UINT16_SEMIPLANAR = 1 << 5,
  DW_CAMERA_OUTPUT_CUDA_YUV444_UINT8_PLANAR = 1 << 6,
  DW_CAMERA_OUTPUT_CUDA_YUV444_UINT16_PLANAR = 1 << 7
}
 Output types supported by the camera. More...
 
enum  dwCameraRawFormat {
  DW_CAMERA_RAW_FORMAT_UNKNOWN = 0,
  DW_CAMERA_RAW_FORMAT_RGGB = 7000,
  DW_CAMERA_RAW_FORMAT_BGGR = 7100,
  DW_CAMERA_RAW_FORMAT_GRBG = 7200,
  DW_CAMERA_RAW_FORMAT_GBRG = 7300,
  DW_CAMERA_RAW_FORMAT_RCCB = 8000,
  DW_CAMERA_RAW_FORMAT_BCCR = 8100,
  DW_CAMERA_RAW_FORMAT_CRBC = 8200,
  DW_CAMERA_RAW_FORMAT_CBRC = 8300,
  DW_CAMERA_RAW_FORMAT_RCCC = 9000,
  DW_CAMERA_RAW_FORMAT_CRCC = 9100,
  DW_CAMERA_RAW_FORMAT_CCRC = 9200,
  DW_CAMERA_RAW_FORMAT_CCCR = 9300,
  DW_CAMERA_RAW_FORMAT_CCCC = 10000,
  DW_CAMERA_RAW_FORMAT_TOF = 20000
}
 Raw encoding formats pixel-order. More...
 
enum  dwCameraType {
  DW_CAMERA_GENERIC = 0,
  DW_CAMERA_GMSL_AR0231 = 3,
  DW_CAMERA_USB_GENERIC = 4,
  DW_CAMERA_GMSL_AR0144 = 5,
  DW_CAMERA_GMSL_AR0138 = 6,
  DW_CAMERA_GMSL_AR0220 = 7,
  DW_CAMERA_GMSL_AR0820 = 8,
  DW_CAMERA_GMSL_MN34906 = 9,
  DW_CAMERA_GMSL_OV2311 = 10,
  DW_CAMERA_GMSL_IMX390 = 11,
  DW_CAMERA_GMSL_CUSTOM = 99
}
 Enum of available camera sensors. More...
 

Functions

DW_API_PUBLIC dwStatus dwSensorCamera_getCUDAStream (cudaStream_t *stream, dwSensorHandle_t sensor)
 Gets the CUDA stream used. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getImage (dwImageHandle_t *image, dwCameraOutputType type, dwCameraFrameHandle_t frame)
 Gets the output image/s image in a format specified by the output type. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getImageAsync (dwImageHandle_t *image, dwCameraOutputType type, dwCameraFrameHandle_t frame)
 Gets the output image/s image in a format specified by the output type. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getImageProperties (dwImageProperties *imageProperties, dwCameraOutputType type, dwSensorHandle_t sensor)
 Gets information about the image properties for a given 'dwCameraImageOutputType'. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getNumSupportedCaptureModes (uint32_t *numModes, dwSensorHandle_t sensor)
 Gets number of supported capture modes. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getNvMediaIPPManager (NvMediaIPPManager **manager, dwSensorHandle_t sensor)
 Gets the NvMediaIPPManager used for GMSL camera IPP setup and event callback. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getSensorProperties (dwCameraProperties *properties, dwSensorHandle_t sensor)
 Gets information about the camera sensor. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getSupportedCaptureMode (dwCameraProperties *captureMode, uint32_t modeIdx, dwSensorHandle_t sensor)
 Gets capture modes by specified index. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_getTimestamp (dwTime_t *timestamp, dwCameraFrameHandle_t frameHandle)
 Gets the timestamp of the current camera frame. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_readFrame (dwCameraFrameHandle_t *frameHandle, dwTime_t timeout_us, dwSensorHandle_t sensor)
 Reads a frame handle from the camera sensor. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_returnFrame (dwCameraFrameHandle_t *frameHandle)
 Returns a frame to the camera after it has been consumed. More...
 
DW_API_PUBLIC dwStatus dwSensorCamera_setCUDAStream (cudaStream_t stream, dwSensorHandle_t sensor)
 Sets the CUDA stream used by getImageAsync during internal cuda related operations. More...
 

Data Structure Documentation

◆ dwCameraProperties

struct dwCameraProperties
Data Fields
dwCameraType cameraType

Type of the camera

dwCameraExposureControl exposure

Exposure control

dwCameraFOV fov

FOV of the lens

float32_t framerate

Framerate in Hz

uint32_t imageBitDepth

Bit depth of image

bool isSimulated

Camera is being simulated

int32_t outputTypes

Or'ed list of available 'dwCameraOutputType'

dwCameraRawFormat rawFormat

Raw bayer pattern

dwVector2ui resolution

Physical resolution of the camera sensor

uint32_t revision

Revision of the camera (0 if info is not available)

uint32_t siblings

Number of sibling frames

Typedef Documentation

◆ dwCameraFrameHandle_t

typedef struct dwCameraFrame* dwCameraFrameHandle_t

Handle to captured frame.

Definition at line 67 of file Camera.h.

◆ NvMediaIPPManager

typedef void NvMediaIPPManager

Definition at line 60 of file Camera.h.

Enumeration Type Documentation

◆ dwCameraExposureControl

Enum of exposure control types.

Enumerator
DW_CAMERA_EXPOSURE_NONE 

No exposure control.

DW_CAMERA_EXPOSURE_UNKNOWN 

Unknown exposure control.

DW_CAMERA_EXPOSURE_AE 

Default exposure control.

DW_CAMERA_EXPOSURE_BAE 

Exposure control with bracketed auto exposure.

DW_CAMERA_EXPOSURE_CUSTOM 

Exposure control using user's custom callback.

Definition at line 132 of file Camera.h.

◆ dwCameraFOV

Enum of available FOV in degrees for camera lenses.

Enumerator
DW_CAMERA_FOV_UNKNOWN 
DW_CAMERA_FOV_30 
DW_CAMERA_FOV_48 
DW_CAMERA_FOV_50 
DW_CAMERA_FOV_55 
DW_CAMERA_FOV_60 
DW_CAMERA_FOV_65 
DW_CAMERA_FOV_90 
DW_CAMERA_FOV_100 
DW_CAMERA_FOV_110 
DW_CAMERA_FOV_120 
DW_CAMERA_FOV_185 

Definition at line 146 of file Camera.h.

◆ dwCameraOutputType

Output types supported by the camera.

DW_CAMERA_OUTPUT_NATIVE_* types return the image directly from the layer underneath as it is represented in system memory, while for non native output types images are converted and streamed through ad hoc streamers.

Enumerator
DW_CAMERA_OUTPUT_NATIVE_PROCESSED 

processed images (usually be YUV420 planar or RGB planar)

DW_CAMERA_OUTPUT_NATIVE_RAW 

raw image

DW_CAMERA_OUTPUT_CUDA_RGBA_UINT8 

for processed images
RGBA image supported in all processed use cases

DW_CAMERA_OUTPUT_CUDA_YUV420_UINT8_PLANAR 

simple yuv420 output, supported in most use cases (see doc)

DW_CAMERA_OUTPUT_CUDA_RAW_UINT16 

for raw images

DW_CAMERA_OUTPUT_CUDA_YUV420_UINT16_SEMIPLANAR 

other YUV processed outputs (see devguide)

DW_CAMERA_OUTPUT_CUDA_YUV444_UINT8_PLANAR 

other YUV processed outputs (see devguide)

DW_CAMERA_OUTPUT_CUDA_YUV444_UINT16_PLANAR 

other YUV processed outputs (see devguide)

Definition at line 72 of file Camera.h.

◆ dwCameraRawFormat

Raw encoding formats pixel-order.

Enumerator
DW_CAMERA_RAW_FORMAT_UNKNOWN 
DW_CAMERA_RAW_FORMAT_RGGB 
DW_CAMERA_RAW_FORMAT_BGGR 
DW_CAMERA_RAW_FORMAT_GRBG 
DW_CAMERA_RAW_FORMAT_GBRG 
DW_CAMERA_RAW_FORMAT_RCCB 
DW_CAMERA_RAW_FORMAT_BCCR 
DW_CAMERA_RAW_FORMAT_CRBC 
DW_CAMERA_RAW_FORMAT_CBRC 
DW_CAMERA_RAW_FORMAT_RCCC 
DW_CAMERA_RAW_FORMAT_CRCC 
DW_CAMERA_RAW_FORMAT_CCRC 
DW_CAMERA_RAW_FORMAT_CCCR 
DW_CAMERA_RAW_FORMAT_CCCC 
DW_CAMERA_RAW_FORMAT_TOF 

Definition at line 93 of file Camera.h.

◆ dwCameraType

Enum of available camera sensors.

Enumerator
DW_CAMERA_GENERIC 

Generic video source, e.g. video file with raw or processed data

DW_CAMERA_GMSL_AR0231 

GMSL AR0231 camera (rev-7)

DW_CAMERA_USB_GENERIC 

Generic USB camera

DW_CAMERA_GMSL_AR0144 

GMSL AR0144 camera

DW_CAMERA_GMSL_AR0138 

GMSL AR0138 camera

DW_CAMERA_GMSL_AR0220 

GMSL AR0220 camera

DW_CAMERA_GMSL_AR0820 

GMSL AR0820 camera

DW_CAMERA_GMSL_MN34906 

GMSL MN34906 camera

DW_CAMERA_GMSL_OV2311 

GMSL OV2311 camera

DW_CAMERA_GMSL_IMX390 

GMSL IMX390 camera

DW_CAMERA_GMSL_CUSTOM 

GMSL custom camera defined using NvSIPL

Definition at line 117 of file Camera.h.

Function Documentation

◆ dwSensorCamera_getCUDAStream()

DW_API_PUBLIC dwStatus dwSensorCamera_getCUDAStream ( cudaStream_t *  stream,
dwSensorHandle_t  sensor 
)

Gets the CUDA stream used.

Parameters
[out]streamReturns the CUDA stream in sensor.
[in]sensorA pointer to the camera handle that is updated.
Returns
DW_INVALID_ARGUMENT: if the sensor handle or cudaStream is null
DW_INVALID_HANDLE: if the sensor handle is not a camera
DW_SUCCESS

◆ dwSensorCamera_getImage()

DW_API_PUBLIC dwStatus dwSensorCamera_getImage ( dwImageHandle_t image,
dwCameraOutputType  type,
dwCameraFrameHandle_t  frame 
)

Gets the output image/s image in a format specified by the output type.

Depending on the type requested, conversion and streaming handled by the camera implicitly might be required. The call is blocking NOTE: the underlying resources are still in the frame handle and the image returned is intended not to be modified. For this reason, any modifications to this 'dwImageHandle_t' or 'dwImageCPU', 'dwImageCUDA', 'dwImageGL' or 'dwImageNvMedia' returned by 'dwImage_getCPU' ('dwImage_getCUDA', 'dwImage_getGL' and 'dwImage_getNvMedia') will result in undefined behavior.

Parameters
[out]imageHandle to the image received by the camera
[in]typeOuptut type of the image. This is represented by a limited useful number of options which can be chosen at runtime
[in]frameCamera frame handle of the captured frame
Returns
DW_CUDA_ERROR - if the underlying camera driver had a CUDA error.
In such case it is not possible to recover a DW_CAMERA_OUTOUT_CUDA_X output, however it is
possible to successfully request a DW_CAMERA_NATIVE_X output and work with it
DW_NVMEDIA_ERROR - if underlying camera driver had an NvMedia error.
DW_INVALID_HANDLE - if given camera frame handle is not valid.
DW_INVALID_ARGUMENT - if given image handle is null.
DW_NOT_IMPLEMENTED - if the method for this image type is not implemented by given camera.
DW_SUCCESS

◆ dwSensorCamera_getImageAsync()

DW_API_PUBLIC dwStatus dwSensorCamera_getImageAsync ( dwImageHandle_t image,
dwCameraOutputType  type,
dwCameraFrameHandle_t  frame 
)

Gets the output image/s image in a format specified by the output type.

Depending on the type requested, conversion and streaming handled by the camera implicitly might be required, which happens on the cudaStream specified at 'dwSensorCamera_setCUDAStream()' NOTE: the underlying resources are still in the frame handle and the image returned is intended not to be modified. For this reason, any modifications to this 'dwImageHandle_t' or 'dwImageCPU', 'dwImageCUDA', 'dwImageGL' or 'dwImageNvMedia' returned by 'dwImage_getCPU' ('dwImage_getCUDA', 'dwImage_getGL' and 'dwImage_getNvMedia') will result in undefined behavior.

Parameters
[out]imageHandle to the image received by the camera
[in]typeOuptut type of the image. This is represented by a limited useful number of options which can be chosen at runtime
[in]frameCamera frame handle of the captured frame
Returns
DW_CUDA_ERROR - if the underlying camera driver had a CUDA error.
In such case it is not possible to recover a DW_CAMERA_OUTOUT_CUDA_X output, however it is
possible to successfully request a DW_CAMERA_NATIVE_X output and work with it
DW_NVMEDIA_ERROR - if underlying camera driver had an NvMedia error.
DW_INVALID_HANDLE - if given handle is not valid.
DW_NOT_IMPLEMENTED - if the method for this image type is not implemented by given camera.
DW_SUCCESS

◆ dwSensorCamera_getImageProperties()

DW_API_PUBLIC dwStatus dwSensorCamera_getImageProperties ( dwImageProperties imageProperties,
dwCameraOutputType  type,
dwSensorHandle_t  sensor 
)

Gets information about the image properties for a given 'dwCameraImageOutputType'.

Parameters
[out]imagePropertiesA pointer to image properties of the frames captured by the camera.
[in]typeFormat of the output image to get the properties of
[in]sensorSensor handle of the camera sensor previously created with dwSAL_createSensor().
Note
The dimensions of the returned properties corresponds to the dimension of the image returned through 'dwSensorCamera_getImage*' methods
Returns
DW_INVALID_ARGUMENT: if the sensor handle is null
DW_INVALID_HANDLE: if the sensor handle is not a camera
DW_NOT_AVAILABLE: when the setup of the sensor is incompatible with the output type requested
DW_SUCCESS

◆ dwSensorCamera_getNumSupportedCaptureModes()

DW_API_PUBLIC dwStatus dwSensorCamera_getNumSupportedCaptureModes ( uint32_t *  numModes,
dwSensorHandle_t  sensor 
)

Gets number of supported capture modes.

Parameters
[out]numModesA pointer to the number of available capture modes.
[in]sensorSensor handle of the camera sensor previously created with dwSAL_createSensor().
Returns
DW_INVALID_ARGUMENT: if the sensor handle is null
DW_INVALID_HANDLE: if the sensor handle is not a camera
DW_SUCCESS

◆ dwSensorCamera_getNvMediaIPPManager()

DW_API_PUBLIC dwStatus dwSensorCamera_getNvMediaIPPManager ( NvMediaIPPManager **  manager,
dwSensorHandle_t  sensor 
)

Gets the NvMediaIPPManager used for GMSL camera IPP setup and event callback.

Parameters
[out]managerA pointer to the NvMediaIPPManager instance created by the sensor.
[in]sensorSensor handle of the camera sensor previously created with dwSAL_createSensor().
Returns
DW_NOT_AVAILABLE if NvMedia is not available.
DW_INVALID_ARGUMENT if given sensor handle or output pointer are NULL
DW_SUCCESS
Note
The ownership of the NvMedia IPP manager remains with the sensor.

◆ dwSensorCamera_getSensorProperties()

DW_API_PUBLIC dwStatus dwSensorCamera_getSensorProperties ( dwCameraProperties properties,
dwSensorHandle_t  sensor 
)

Gets information about the camera sensor.

Parameters
[out]propertiesA pointer to the properties of the camera.
[in]sensorSensor handle of the camera sensor previously created with dwSAL_createSensor().
Returns
DW_INVALID_ARGUMENT: if the sensor handle is null
DW_INVALID_HANDLE: if the sensor handle is not a camera
DW_SUCCESS

◆ dwSensorCamera_getSupportedCaptureMode()

DW_API_PUBLIC dwStatus dwSensorCamera_getSupportedCaptureMode ( dwCameraProperties captureMode,
uint32_t  modeIdx,
dwSensorHandle_t  sensor 
)

Gets capture modes by specified index.

Parameters
[out]captureModeA pointer to available capture mode.
[in]modeIdxIndex of a mode to retrieve.
[in]sensorSensor handle of the camera sensor previously created with dwSAL_createSensor().
Returns
DW_INVALID_ARGUMENT: if the sensor handle is null
DW_INVALID_HANDLE: if the sensor handle is not a camera
DW_SUCCESS

◆ dwSensorCamera_getTimestamp()

DW_API_PUBLIC dwStatus dwSensorCamera_getTimestamp ( dwTime_t timestamp,
dwCameraFrameHandle_t  frameHandle 
)

Gets the timestamp of the current camera frame.

Parameters
[out]timestampThe timestamp of the current camera frame.
[in]frameHandleHandle to a captured frame.
Returns
DW_INVALID_HANDLE - if given handle is not valid.
DW_SUCCESS

◆ dwSensorCamera_readFrame()

DW_API_PUBLIC dwStatus dwSensorCamera_readFrame ( dwCameraFrameHandle_t frameHandle,
dwTime_t  timeout_us,
dwSensorHandle_t  sensor 
)

Reads a frame handle from the camera sensor.

The reading is a blocking call. With the frame handle, the associated data can be queried from the sensor. Available data is configured during sensor creation.

Parameters
[out]frameHandleA pointer to a handle to a frame read from the camera. With the handle, different data can be queried. The frame handle must be returned to be put back into the internal pool.
[in]timeout_usTimeout in microseconds to wait for a new frame. Special values: DW_TIMEOUT_INFINITE - to wait infinitly. Zero - means polling of internal queue.
[in]sensorSensor handle of the camera previously created with 'dwSAL_createSensor()'.
Returns
DW_CUDA_ERROR - if the underlying camera driver had a CUDA error.
DW_NVMEDIA_ERROR - if underlying camera driver had an NvMedia error.
DW_INVALID_HANDLE - if given sensor handle is not valid.
DW_INVALID_ARGUMENT - if given frame pointer is NULL
DW_NOT_IMPLEMENTED - if the method for this image type is not implemented by given camera.
DW_TIME_OUT - if no frame could be acquired within given time interval.
This is a recoverable error if the following behavior is observed: the expected rate
at which frames are returned is a single frame every 1/fps seconds, therefore a timeout
should be set such as to cover one frame instance. Frames in a live camera are not returned
consistently therefore a time delta duration is to be expected close to the rate (ie +-10%).
If the total timeouts that have been accummulated after all retries amount to 3 frames interval
the camera is to be considered unresponsive.
DW_NOT_AVAILABLE - if sensor has not been started or data is not available in polling mode.
DW_NOT_READY - if sensor is stopped or not started or has started but hasn't begun acquiring frames after the specified timeout. If the sensor is in a stopped or not started state, the sensor should be started with dwSensor_start. After that The status will be returned until the first frame acquired. If this doesn't happen within 2 seconds, the sensor is to be considered unresponsive.
DW_SAL_SENSOR_ERROR - if there was an i/o error.
DW_END_OF_STREAM - if end of stream reached.
DW_BUFFER_FULL - if there are no more available frames to be read. To recover, return frames to free buffer space
DW_SUCCESS

◆ dwSensorCamera_returnFrame()

DW_API_PUBLIC dwStatus dwSensorCamera_returnFrame ( dwCameraFrameHandle_t frameHandle)

Returns a frame to the camera after it has been consumed.

All data associated with this handle is invalid after the handle has been returned.

Parameters
[in]frameHandleHandle previously read from the camera to be returned to the pool.
Returns
DW_CUDA_ERROR - if the underlying camera driver had a CUDA error.
DW_NVMEDIA_ERROR - if underlying camera driver had NvMedia error.
DW_INVALID_ARGUMENT - if given handle is NULL
DW_INVALID_HANDLE - if given handle is not valid.
DW_CALL_NOT_ALLOWED - if the sensor this frame camera from has already been released.
DW_SUCCESS

◆ dwSensorCamera_setCUDAStream()

DW_API_PUBLIC dwStatus dwSensorCamera_setCUDAStream ( cudaStream_t  stream,
dwSensorHandle_t  sensor 
)

Sets the CUDA stream used by getImageAsync during internal cuda related operations.

Parameters
[in]streamThe CUDA stream to use.
[in]sensorA pointer to the camera handle that is updated.
Returns
DW_INVALID_ARGUMENT: if the sensor handle is null
DW_INVALID_HANDLE: if the sensor handle is not a camera
DW_CUDA_ERROR: when the cudaStream cannot be set
DW_SUCCESS