NvMedia Surface
This topic explains how to use the NvMedia Surface API.
NvMedia surfaces are data structures (“surfaces”) that NvMedia creates in SoC DRAM memory for storing image or video pixel data.
This topic assumes that you have a basic understanding of NvSciBuf. See the NvSciBuf User Guide for more information.
Types of Surfaces
NvMedia supports two types of surfaces:
• NvMedia image surfaces are used with NvMedia image components.
NvMedia image surfaces can be created in two ways:
• With NvMediaImageCreateNew(). You can use this method in HMI and IVI applications where deterministic behavior is not essential.
• With the NvMedia NvSciBuf API. NVIDIA recommends this method for Advanced Driver Assistance Systems (ADAS) and autonomous application development. It also facilitates data sharing between NvMedia and NVIDIA® CUDA®.
• NvMedia Video surfaces: With NvMediaVideoSurfaceCreateNew().
Types of Surface AttributesNvMedia surfaces have two types of attributes:
• Surface format attributes describe a surface’s physical layout and pixel color format in SoC DRAM memory
• Surface allocation attributes describe additional properties of a surface, such as:
• Width, height, and other properties such as image sensor embedded line information)
• CPU access mapping (cached/uncached/unmapped)
• Shared memory space across Virtual Machine partitions
• Allocation policy for operations with special quality of service (QoS) requirements, such as streaming pixel data capture to memory or display from memory
• Scan type (progressive or interlaced)
Surface Format Attributes
The following sections describe the surface format attributes that NvMedia surfaces may have.
NVM_SURF_ATTR_SURF_TYPE Attribute
This attribute specifies the class of pixel color format. The value may be:
• NVM_SURF_ATTR_SURF_TYPE_YUV
Indicates one of the YUV formats, represented by the YUV color space.
Pixel color components are Y (Luminance), U and V (Chrominance), A (Alpha).
• NVM_SURF_ATTR_SURF_TYPE_RGBA
Indicates one of the RGBA formats, represented by the RGB color space.
Pixel color components are R (Red), G (Green), B (Blue), and A (Alpha).
• NVM_SURF_ATTR_SURF_TYPE_RAW
Indicates the Bayer pixel format represented as a 2×2 Bayer color filter array.
NVM_SURF_ATTR_LAYOUT Attribute
Specifies the storage layout of the pixels. The value may be:
• NVM_SURF_ATTR_LAYOUT_PL
Indicates pitch-linear layout, using pitch-linear mapping, in which pixels are assigned incrementing addresses across each successive row of the image. A surface’s pixel addresses are calculated by:
address = offset + pitch * py + pixel * px
Where:
• address is the byte address of a pixel in the surface
• offset is the address of the surface
• pitch is the number of bytes occupied by the pixels in a line
• py is the pixel’s Y coordinate
• pixel is the number of bytes occupied by a pixel
• px is the pixel’s X coordinate
Although pitch addressing is standard across most hardware and software platforms, it limits performance because it has poor memory locality, due to the fact that each line in a surface image generally maps to a different DRAM page.
• NVM_SURF_ATTR_LAYOUT_BL
Indicates block-linear layout. This layout is similar to pitch-linear layout, but maps blocks of pixels using pitch-linear addressing, rather than individual pixels. This layout provides spatial locality for addressing, and improves memory access performance. Pixel data layout within a block is an internal detail of the NVIDIA SoC design, and varies across NVIDIA SoC architectures.
NVM_SURF_ATTR_BITS_PER_COMPONENT Attribute
Specifies the width of each color component present in the pixel color format. The value may be:
• NVM_SURF_ATTR_BITS_PER_COMPONENT_8
Indicates that each color component present is 8 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_10
Indicates that each color component present is 10 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_12
Indicates that each color component present is 12 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_14
Indicates that each color component present is 14 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_16
Indicates that each color component present is 16 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_32
Indicates that each color component present is 32 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_LAYOUT_16_8_8
Indicates a layout of three color components in which the color components are respectively 16, 8, and 8 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_LAYOUT_10_8_8
Indicates a layout of three color components in which the color components are respectively 10, 8, and 8 bits wide.
• NVM_SURF_ATTR_BITS_PER_COMPONENT_LAYOUT_2_10_10_10
Indicates a layout of four color components in which the color components are respectively 2, 10, 10, and 10 bits wide.
NVM_SURF_ATTR_DATA_TYPE Attribute
Specifies the data type of the color components. The value may be:
• NVM_SURF_ATTR_DATA_TYPE_UINT
Indicates that each color component value is an unsigned normalized integer ranging from 0 to the maximum value representable by the width of the components (as specified by the NVM_SURF_ATTR_BITS_PER_COMPONENT).
• NVM_SURF_ATTR_DATA_TYPE_INT
Indicates that each color component value is a signed two's complement integer. The width of the component includes the sign bit.
• NVM_SURF_ATTR_DATA_TYPE_FLOAT
Indicates that each color component value is a float.
NVM_SURF_ATTR_DATA_TYPE, in combination with NVM_SURF_ATTR_BITS_PER_COMPONENT, determines the data format of an individual color component.
Example float combinations are:
• An IEEE 754 single precision float (binary32) with 1 sign bit, 8 exponent bits and 23 mantissa bits:
NVM_SURF_ATTR_DATA_TYPE = NVM_SURF_ATTR_DATA_TYPE_FLOAT, NVM_SURF_ATTR_BITS_PER_COMPONENT = NVM_SURF_ATTR_BITS_PER_COMPONENT_32:
• An IEEE 754 half-precision float (binary16) with 1 sign bit, 5 exponent bits and 10 mantissa bits:
NVM_SURF_ATTR_DATA_TYPE = NVM_SURF_ATTR_DATA_TYPE_FLOAT, NVM_SURF_ATTR_BITS_PER_COMPONENT = NVM_SURF_ATTR_BITS_PER_COMPONENT_16:
NVM_SURF_ATTR_MEMORY Attribute
Specifies the memory layout of a color component present in the pixel color format. The value may be:
• NVM_SURF_ATTR_MEMORY_PACKED
Indicates that the components present are stored interleaved in the same plane.
• NVM_SURF_ATTR_MEMORY_PLANAR
Indicates that the components present are stored in separate planes (one plane per component).
• NVM_SURF_ATTR_MEMORY_SEMI_PLANAR
Applicable only for YUV pixel formats. Indicates that there are two planes instead of the usual three, one for luminance and one for both chrominance components. Chrominance components are stored in interleaved order.
NVM_SURF_ATTR_SUB_SAMPLING_TYPE Attribute
Applicable only to YUV pixel formats. Represents the A: B: C notation that is used to describe how often U and V are sampled horizontally and vertically relative to Y. The value may be:
• NVM_SURF_ATTR_SUB_SAMPLING_TYPE_444
Indicates no horizontal and vertical downsampling of chrominance components U and V relative to Y.
• NVM_SURF_ATTR_SUB_SAMPLING_TYPE_420
Indicates 2:1 horizontal downsampling and 2:1 vertical downsampling.
• NVM_SURF_ATTR_SUB_SAMPLING_TYPE_422
Indicates 2:1 horizontal downsampling and no vertical downsampling.
• NVM_SURF_ATTR_SUB_SAMPLING_TYPE_422R
Indicates no horizontal downsampling and 2:1 vertical downsampling.
NVM_SURF_ATTR_ COMPONENT_ORDER Attribute
Specifies the order of color components in memory.
For packed pixel formats, represents the order of color components from least significant bit to most significant bit.
• For planar and semi-planar pixel formats, represents the order of the planes containing the color components.
• For RAW pixel formats, represents the arrangement of color components in the 2×2 Bayer color filter pattern.
The value may be:
• NVM_SURF_ATTR_COMPONENT_ORDER_RGBA
Indicates packed RGBA pixel format with the four components R, G, B and A arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_ARGB
Indicates packed RGBA pixel format with the four components A, R, G and B arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_BGRA
Indicates packed RGBA pixel format with the four components B, G, R and A arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_ALPHA
Indicates packed RGBA pixel format with a single Alpha component.
• NVM_SURF_ATTR_COMPONENT_ORDER_RG
Indicates packed RGBA pixel format with dual components R and G arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_YUV
Indicates YUV pixel format with the three components: Y, U, and V, arranged in that order as separate planes for planar formats.
• Y is in a separate plane.
• U and V are in single plane, arranged consecutively from least significant bit to most significant bit.
• NVM_SURF_ATTR_COMPONENT_ORDER_YVU
Indicates YUV pixel format with three components.
• Y, V, and U arranged in that order as separate planes for planar formats.
• Y in a separate plane and V and U in a single plane, arranged consecutively from least significant bit to most significant bit.
• NVM_SURF_ATTR_COMPONENT_ORDER_LUMA
Indicates packed YUV pixel format with a single Luminance component.
• NVM_SURF_ATTR_COMPONENT_ORDER_XYUV
Indicates packed XYUV pixel format with the four components X, Y, U, and V arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_YUYV
Indicates packed YUYV pixel format with the four components Y, U, Y, and V arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_YVYU
Indicates packed YUYV pixel format with the four components Y, V, Y, and U arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_VYUY
Indicates packed YUYV pixel format with the four components V, Y, U, and Y arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_UYVY
Indicates packed YUYV pixel format with the four components U, Y, V, and Y arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_XUYV
Indicates packed XYUV pixel format with the four components X, U, Y, and V arranged consecutively from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_RGGB
Indicates packed 2×2 Bayer RGB pixel format with R, G, and G, B pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_BGGR
Indicates packed 2×2 Bayer RGB pixel format with B, G, and G, R pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_GRBG
Indicates packed 2×2 Bayer RGB pixel format with G, R, and B, G pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_GBRG
Indicates packed 2×2 Bayer RGB pixel format with G, B and R, G pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_RCCB
Indicates packed 2×2 Bayer RCB pixel format (C represents clear component), with R, C, and C, B pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_BCCR
Indicates packed 2×2 Bayer RCB pixel format (C represents clear component), with B, C, and C, R pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_CRBC
Indicates packed 2×2 Bayer RCB pixel format (C represents clear component), with C, R, and B, C pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_CBRC
Indicates packed 2×2 Bayer RCB pixel format (C represents clear component), with C, B, and R, C pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_RCCC
Indicates packed 2×2 Bayer RC1C2 pixel format (C1 and C2 represent clear components), with R, C1, and C1, C2 pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_CCCR
Indicates packed 2×2 Bayer RC1C2 pixel format (C1 and C2 represent clear components), with C2, C1, and C1, R pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_CRCC
Indicates packed 2×2 Bayer RC1C2 pixel format (C1 and C2 represent clear components), with C1, R, and C2, C1 pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
• NVM_SURF_ATTR_COMPONENT_ORDER_CCRC
Indicates packed 2×2 Bayer RC1C2 pixel format (C1 and C2 represent clear components), with C1, C2, and R, C1 pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory.
Surface Allocation Attributes
The following sections describe the surface allocation attributes that NvMedia surfaces may have.
NVM_SURF_ATTR_WIDTH and NVM_SURF_ATTR_HEIGHT Attributes
Specifies the width and height of the surface to be allocated.
NVM_SURF_ATTR_EMB_LINES_TOP and NVM_SURF_ATTR_EMB_LINES_BOTTOM Attributes
Specifies the number of additional lines present in the surface such as image sensor embedded line information.
NVM_SURF_ATTR_CPU_ACCESS Attribute
Specifies the coherency policy to be used for accesses of the surface from the CPU. The value may be:
• NVM_SURF_ATTR_CPU_ACCESS_UNCACHED
Specifies that accesses from CPU never cache data.
Setting this attribute results in the following behavior: While writing to the surface from the CPU using NvMediaImageLock() and NvMediaImageUnlock(), NvMedia uses appropriate memory barriers before handing over the surface to hardware engines to ensure coherency.
• NVM_SURF_ATTR_CPU_ACCESS_CACHED
Specifies that accesses from the CPU can pass through caches and store buffers.
Setting this attribute results in the following behavior:
• While reading the surface from the CPU using NvMediaImageLock() and NvMediaImageUnlock(), caches are invalidated as necessary to ensure that the CPU gets the latest data written by the hardware engines.
• While writing the surface from the CPU using NvMediaImageLock() and NvMediaImageUnlock(), caches are flushed as necessary before handing over the surface to hardware engines to ensure coherency.
In both the above cases, the surface is mapped and it can be accessed with a mapping into the current process’s virtual address space.
• NVM_SURF_ATTR_CPU_ACCESS_UNMAPPED
Specifies a coherency policy that is the same as for NVM_SURF_ATTR_CPU_ACCESS_UNCACHED. However, the surface is not mapped into the current process’s virtual address space.
If the attribute is not specified, the coherency policy defaults to NVM_SURF_ATTR_CPU_ACCESS_UNCACHED.
NVM_SURF_ATTR_ALLOC_TYPE Attribute
Specifies the allocation policy for hardware units with special QoS requirements, such ICP (for data capture to memory) and IDP (for display from memory).
• NVM_SURF_ATTR_ALLOC_TYPE_ISOCHRONOUS
Applicable only in a virtualized environment. Specifies that the surface is allocated in:
• Carveout memory heap in external memory if the surface doesn’t have to be shared with other virtual-machine partitions
• Inter-partition heap if the surface does need to be shared across two virtual-machine partitions (specified by NVM_SURF_ATTR_PEER_VM_ID).
• NVM_SURF_ATTR_ALLOC_TYPE_SECURED
Specifies that the surface is allocated in the Video Protection Region (VPR) heap.
If the attribute is not specified, the surface is allocated in the IOMMU heap.
NVM_SURF_ATTR_PEER_VM_ID Attribute
Applicable only in a virtualized environment. Specifies the virtual machine partition ID with which the allocated surface is to be shared.
NVM_SURF_ATTR_SCAN_TYPE Attribute
Specifies the surface display scan type. The value may be:
• NVM_SURF_ATTR_SCAN_TYPE_PROGRESSIVE
Indicates the progressive scan display type, in which the entire surface is refreshed every cycle.
• NVM_SURF_ATTR_SCAN_TYPE_INTERLACED
Indicates the interlaced scan display type, in which alternate horizontal pixel rows are refreshed in one cycle and the remaining rows are refreshed in the next, so that two complete scans are required to display the entire surface.
If the attribute is not specified, it defaults to NVM_SURF_ATTR_SCAN_TYPE_PROGRESSIVE.
Surface Format Functions
This section describes NvMedia Surface API functions that convert between a format type and a set of input surface format attributes.
NvMediaSurfaceFormatGetType()
Returns the NvMedia surface format type for a specified set of surface format attributes.
You can use these macros to set surface format attributes and define pixel color format conveniently and efficiently.
NVM_SURF_FMT_SET_ATTR_YUV(attr, #COMPONENT_ORDER,
#SUB_SAMPLING_TYPE,
#MEMORY, #DATA_TYPE,
#BITS_PER_COMPONENT,
#LAYOUT);
NVM_SURF_FMT_SET_ATTR_RGBA(attr, #COMPONENT_ORDER,
#DATA_TYPE,
#BITS_PER_COMPONENT,
#LAYOUT);
NVM_SURF_FMT_SET_ATTR_RAW(attr, #COMPONENT_ORDER,
#DATA_TYPE,
#BITS_PER_COMPONENT,
#LAYOUT);
NvMediaSurfaceFormatGetAttrs()
Returns the NvMedia surface format attributes for a specified surface type.
Surface Allocation Functions
This section describes NvMedia Surface API functions that create, destory, and manage surfaces.
NvMedia Image Functions
These API functions manage images.
NvMediaImageCreateNew()
Allocates the NvMedia image surface for a given surface type obtained with NvMediaSurfaceFormatGetType() and a given set of surface allocation attributes.
This is an example of code that performs NvMedia image allocation using NvMediaImageCreateNew().
/* Define an array of NvMediaSurfFormatAttr */
NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
/* Set attributes that uniquely represent a YUV 444 8-bit pitch-linear
planar surface. */
NVM_SURF_FMT_SET_ATTR_YUV (surfFormatAttrs, YUV, 444, PLANAR, UINT, 8, PL);
/* Get NvMediaSurfaceType for the specified surface format attributes. */
type = NvMediaSurfaceFormatGetType(surfFormatAttrs,
NVM_SURF_ALLOC_ATTR_MAX);
/* Define an array of NvMediaSurfAllocAttr */
NvMediaSurfAllocAttr surfAllocAttrs[8];
/* Set surface allocation attributes */
surfAllocAttrs[0].type = NVM_SURF_ATTR_WIDTH;
surfAllocAttrs[0].value = 1280;
surfAllocAttrs[1].type = NVM_SURF_ATTR_HEIGHT;
surfAllocAttrs[1].value = 1080;
surfAllocAttrs[2].type = NVM_SURF_ATTR_CPU_ACCESS;
surfAllocAttrs[2].value = NVM_SURF_ATTR_CPU_ACCESS_CACHED;
numSurfAllocAttrs = 3;
/* Call NvMediaImageCreateNew API for surface creation */
image = NvMediaImageCreateNew(device,
type,
surfAllocAttrs,
numSurfAllocAttrs,
0);
NvMediaImageCreateFromNvSciBuf()
Creates an NvMedia image from an NvSciBuf created with the NvSciBuf API.
Every hardware engine in an NVIDIA SoC can have a different alignment or stride constraints. Hence, sharing a buffer across various engines requires that buffer allocation satisfy the constraints of all of the engines that share the buffer. An engine whose constraints are not satisfied may fail to operate on the buffer. The allocation functions provided by the various NvMedia drivers only satisfy the constraints of the engines that are visible to them, and so cannot be used to allocate shared buffers.
NvSciBuf is a buffer allocation module that satisfies a common set of constraints that are compatible with all of the hardware engines. It thus can allocate buffers that are shareable across the hardware engines visible to various drivers.
This is a typical flow to allocate an NvSciBufObj, which can be mapped to an NvMediaImage:
1. The application creates an NvSciBufAttrList.
2. The application queries NvMedia to fill the NvSciBufAttrList by passing a set of NvMediaImage allocation attributes and an NvMediaSurfaceType as input to NvMediaImageFillNvSciBufAttrs().
3. The application may set any of the public NvSciBufAttribute values that NvMedia does not set.
For more details on NvSciBuf concepts, terminology, and the API, see NvSciBuf User Guide.
The following NvSciBuf input attributes are set by NvMedia, and so must not be set by the application:
• NvSciBufGeneralAttrKey_Types
• NvSciBufGeneralAttrKey_NeedCpuAccess
• NvSciBufGeneralAttrKey_EnableCpuCache
• NvSciBufImageAttrKey_Layout
• NvSciBufImageAttrKey_PlaneCount
• NvSciBufImageAttrKey_TopPadding
• NvSciBufImageAttrKey_BottomPadding
• NvSciBufImageAttrKey_LeftPadding
• NvSciBufImageAttrKey_RightPadding
• NvSciBufImageAttrKey_PlaneColorFormat
• NvSciBufImageAttrKey_PlaneColorStd
• NvSciBufImageAttrKey_PlaneBaseAddrAlign
• NvSciBufImageAttrKey_PlaneWidth
• NvSciBufImageAttrKey_PlaneHeight
• NvSciBufImageAttrKey_PlaneScanType
• NvSciBufImageAttrKey_VprFlag
The following image attributes are not set by NvMedia and so must be set by the application:
• NvSciBufGeneralAttrKey_RequiredPerm
4. If the same NvSciBufObj object has to be shared with other user mode drivers (UMDs), the application can get the corresponding NvSciBufAttrList from the respective UMDs.
5. The application asks NvSciBuf to reconcile all of the filled NvSciBufAttrList objects, then allocate an NvSciBuf Object.
6. The application queries NvMedia to create an NvMediaImage from the allocated NvSciBuf Object by calling NvMediaImageCreateFromNvSciBuf().
7. The NvMediaImage can be passed as input and output to any of the NvMedia API functions that accept an NvMediaImage as a parameter.
NvMedia Image Surface Allocation with NvSciBuf
This is an example of how to allocate an NvMedia image surface with NvSciBuf:
NvMediaDevice *device;
NvMediaStatus status;
NvSciError err;
NvSciBufModule module;
NvSciBufAttrList attrlist, conflictlist;
NvSciBufObj bufObj;
NvMediaImage *image;
NvMediaSurfaceType nvmsurfacetype;
NvMediaSurfAllocAttr surfAllocAttrs[8];
NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
/*NvMedia related initialization. */
device = NvMediaDeviceCreate();
status = NvMediaImageNvSciBufInit();
/*NvSciBuf related initialization. */
err = NvSciBufModuleOpen(&module);
/*Create NvSciBuf attribute list. */
err = NvSciBufAttrListCreate(module, &attrlist);
/* Initialize surfFormatAttrs and surfAllocAttrs as required. */
/* Define an array of NvMediaSurfFormatAttr */ NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
/* Set attributes that uniquely represent a YUV 444 8-bit pitch-linear planar surface */
NVM_SURF_FMT_SET_ATTR_YUV (surfFormatAttrs, YUV, 444, PLANAR, UINT, 8, PL);
/* Get NvMediaSurfaceType for the specified surface format attributes. */
type = NvMediaSurfaceFormatGetType(surfFormatAttrs,
NVM_SURF_ALLOC_ATTR_MAX);
/* Define an array of NvMediaSurfAllocAttr objects. */
NvMediaSurfAllocAttr surfAllocAttrs[8];
/* Set surface allocation attributes. */
surfAllocAttrs[0].type = NVM_SURF_ATTR_WIDTH;
surfAllocAttrs[0].value = 1280;
surfAllocAttrs[1].type = NVM_SURF_ATTR_HEIGHT;
surfAllocAttrs[1].value = 1080;
surfAllocAttrs[2].type = NVM_SURF_ATTR_CPU_ACCESS;
surfAllocAttrs[2].value = NVM_SURF_ATTR_CPU_ACCESS_CACHED;
numSurfAllocAttrs = 3;
/* Get NvMediaSurfaceType. */
nvmsurfacetype = NvMediaSurfaceFormatGetType(surfFormatAttrs, NVM_SURF_FMT_ATTR_MAX);
/* Ask NvMedia to fill NvSciBufAttrs corresponding to nvmsurfacetype
and surfAllocAttrs. */
status = NvMediaImageFillNvSciBufAttrs(device, nvmsurfacetype, surfAllocAttrs, numsurfallocattrs, 0, attrlist);
/* Reconcile the NvSciBufAttrs and then allocate an NvSciBufObj. */
err = NvSciBufAttrListReconcileAndObjAlloc(&attrlist, 1, bufobj, &conflictlist);
/* Create NvMediaImage from NvSciBufObj. */
status = NvMediaImageCreateFromNvSciBuf(device, bufobj, &image);
/* Free the NvSciBufAttrList which is no longer required. */
err = NvSciBufAttrListFree(attrlist);
/* Use the image as input or output for any of the NvMedia
component. */
....
....
/* Free the resources after use. */
/* Destroy NvMediaImage. */
NvMediaImageDestroy(image);
/* NvMedia related Deinit. */
NvMediaImageNvSciBufDeinit();
NvMediaDeviceDestroy(device);
/* NvSciBuf related deinit. */
NvSciBufObjFree(bufobj);
NvSciBufModuleClose(module);
NvMedia Video
These API functions manage video content.
NvMediaVideoSurfaceCreateNew()
Allocates an NvMedia video surface for the given surface type obtained by a call to NvMediaSurfaceFormatGetType() and the given surface allocation attributes.
NvMedia Video Surface Allocation Example
/* Define an array of NvMediaSurfFormatAttr objects. */
NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
/* Set attributes that uniquely represent a YUV 444 8-bit pitch-linear
planar surface. */
NVM_SURF_FMT_SET_ATTR_YUV (surfFormatAttrs, YUV, 444, PLANAR, UINT, 8, PL);
/* Get NvMediaSurfaceType for the specified surface format attributes. */
type = NvMediaSurfaceFormatGetType(surfFormatAttrs,
NVM_SURF_ALLOC_ATTR_MAX);
/* Define an array of NvMediaSurfAllocAttr objects. */
NvMediaSurfAllocAttr surfAllocAttrs[8];
/* Set surface allocation attributes. */
surfAllocAttrs[0].type = NVM_SURF_ATTR_WIDTH;
surfAllocAttrs[0].value = 1280;
surfAllocAttrs[1].type = NVM_SURF_ATTR_HEIGHT;
surfAllocAttrs[1].value = 1080;
surfAllocAttrs[2].type = NVM_SURF_ATTR_CPU_ACCESS;
surfAllocAttrs[2].value = NVM_SURF_ATTR_CPU_ACCESS_CACHED;
numSurfAllocAttrs = 3;
/* Call NvMediaVideoSurfaceCreateNew() for surface creation. */
vidSurf = NvMediaVideoSurfaceCreateNew(device,
type,
surfAllocAttrs,
numSurfAllocAttrs,
0);
Supported Pixel Formats in NvMedia
Note: | On platforms that have the NVIDIA® AGX Xavier™ SoC, NvMedia supports only UINT surfaces for RAW capture. Using an INT surface for RAW capture will result in an error. |
The following table describes the NvMedia supported pixel formats.
Pixel Format | Description |
R8G8B8A8 | RGBA 8-bit packed normalized integer color format with the four components R/G/B/A) arranged consecutively from least significant bit to most significant bit in memory. NVM_SURF_FMT_SET_ATTR_RGBA (attr, RGBA, UINT, 8, [PL|BL]); |
R8_I R16_I R32_I | RGBA 8, 16, or 32-bit packed signed integer color format with one Alpha component. NVM_SURF_FMT_SET_ATTR_RGBA (attr, ALPHA, INT, [8|16|32], [PL|BL]); |
R16G16_I | RGBA 8-bit packed signed integer color format with two components (R and G) arranged consecutively from least significant bit to most significant bit in memory. NVM_SURF_FMT_SET_ATTR_RGBA (attr, RG, INT, 16, [PL|BL]); |
Y8_U8_V8_N420 Y8_U8_V8_N444 Y8_U8_V8_N422 | YUV [4:2:0|4:4:4|4:2:2] planar 8-bit normalized integer color format with Y, U and V components arranged as separate planes in that order. NVM_SURF_FMT_SET_ATTR_YUV (attr, YUV, [420|422|444], PLANAR, UINT, 8, [PL|BL]); |
Y8_U8V8_N420 Y10_U10V10_N420 Y12_U12V12_N420 | YUV 4:2:0 semi-planar 8, 10, or 12-bit normalized integer color format with Y in a separate plane and U and V arranged consecutively from least significant bit to most significant bit in single plane. NVM_SURF_FMT_SET_ATTR_YUV (attr, YUV, 420, SEMI_PLANAR, UINT, [8|10|12], [PL|BL]); |
Y8_U8V8_N444 Y10_U10V10_N444 Y12_U12V12_N444 | YUV 4:4:4 semi-planar 8, 10, or 12-bit normalized integer color format with Y in a separate plane and U and V arranged consecutively from least significant bit to most significant bit in single plane. NVM_SURF_FMT_SET_ATTR_YUV (attr, YUV, 444, SEMI_PLANAR, UINT, [8|10|12], [PL|BL]); |
Y8U8Y8V8_N422 | YUYV 4:2:2 packed 8-bit normalized integer color format with four components (Y/U/Y/V) arranged consecutively from least significant bit to most significant bit in memory. NVM_SURF_FMT_SET_ATTR_YUV (attr, YUYV, 422, PACKED, UINT, 8, [PL|BL]); |
X16Y16U16V16_F | XYUV Packed FP16 Color Format with the four components (X/Y/U/V) arranged consecutively from least significant bit to most significant bit in memory. NVM_SURF_FMT_SET_ATTR_YUV (attr, XYUV, NONE, PACKED, FLOAT, 16, [PL|BL]); |
Y8 Y10 Y12 Y16 | YUV packed 8, 10, 12, or 16-bit normalized integer color format with one Luminance component. NVM_SURF_FMT_SET_ATTR_YUV (attr, LUMA, NONE, PACKED, UINT, [8|10|12|16], [PL|BL]); |
Bayer RGGB Format RAW8 RAW10 RAW12 RAW16 RAW20 | 2×2 Bayer RGB packed 8, 10, 12, 16, or 20-bit signed integer color format with [R,G|B,G|G,R|G,B] and [G,B|G,R|B,G|R,G] pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory. NVM_SURF_FMT_SET_ATTR_RAW (attr, [RGGB|BGGR|GRBG|GBRG], [UINT|INT], [8|10|12|16|20], [PL|BL]); For UINT or INT platform limitations, see the note at the beginning of Supported Pixel Formats in NvMedia (this section). |
Bayer RCCB Format RAW12 | 2×2 Bayer RCB packed 12-bit signed integer color format with [R,C|B,C|C,R|C,B] and [C,B|C,R|B,C|R,C] pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory. NVM_SURF_FMT_SET_ATTR_RAW (attr, [RCCB|BCCR|CRBC|CBRC], [UINT|INT], 12, [PL|BL]); For UINT or INT platform limitations, see the note at the beginning of Supported Pixel Formats in NvMedia (this section). |
Bayer RCCC Format RAW12 | 2×2 Bayer RCC packed 12-bit signed integer color format with [R,C1|C2,C1|C1,C2|C1,C2] and [C1,C2|C1,C2|C2,C1|C2,C1] pairs arranged in consecutive scan lines from least significant bit to most significant bit in memory. NVM_SURF_FMT_SET_ATTR_RAW (attr, [RCCC|CCCR|CRCC|CCRC], [UINT|INT], 12, [PL|BL]); For UINT or INT platform limitations, see the note at the beginning of Supported Pixel Formats in NvMedia (this section). |
Surface Read Code Examples
These pseudocode examples illustrate CPU memory access, an operation that is expensive and should be used only for debugging.
NvMedia Image Pitch-Linear
This example reads an image surface that represents a YUV 444 8-bit pitch-linear planar surface.
NvMediaImageSurfaceMap surfaceMap;
NvMediaStatus status;
NvMediaImageLock(image,
NVMEDIA_IMAGE_ACCESS_READ,
&surfaceMap);
for ( i = 0; i < surfaceMap.height; i++) {
for ( j = 0; j < surfaceMap.width; j++) {
/* Pitch-linear addressing */
index = i*surfaceMap.surface[0].pitch + j;
...
Read surfaceMap.surface[0].mapping[index] /* Y plane */
Read surfaceMap.surface[1].mapping[index] /* U plane */
Read surfaceMap.surface[2].mapping[index] /* V plane */
...
}
}
NvMedia Image Block-Linear
This example reads an image surface that represents a YUV 444 8-bit block-linear planar surface.
NvMediaImageSurfaceMap surfaceMap;
NvMediaStatus status;
NvMediaImageLock(image,
NVMEDIA_IMAGE_ACCESS_READ,
&surfaceMap);
status = NvMediaImageGetBits(image,
NULL,
(void **) *buf,
*pitches);
