V4L2 Capture Operator

V4L2 camera capture for Holoscan SDK 5.x on Linux. See the module README for SDK setup and build instructions. For Python setup, see the Camera Python guide. For the Python operator interface, see the V4L2 binding section.

C++ reference application

See the V4L2 application guide for graph validation, live capture, memory placement options, and CTest commands.

Python reference application

See the V4L2 Python application guide for validate, capture, and Holoviz preview modes, environment setup, and application tests.

Interface

V4l2CaptureOp opens /dev/video*, negotiates YUYV, streams via mmap, and publishes frames from a reader thread that wakes the operator through a notification source.

The published frame is the packed YUYV 4:2:2 frame, chroma included, under ImageEncoding_YUYV. The tensor is rank 2 and one byte per element, shaped [height, 2 * width]: a row is its own byte extent, and the pixel geometry is read from the descriptor’s width, height, and encoding rather than inferred from the shape. A downstream consumer must declare that carriage, including the placement the operator was constructed with:

spec.input(frame_in, "frame_in")
    .queue_depth(4U)
    .expects_tensor(holoscan::TensorInputSpec{
        .representation = {.memory_kind = holoscan::MemoryKind::kHost,
                           .dtype = holoscan::camera::kFrameElementDtype,
                           .rank = 2U},
    });

Capture width must be even. A packed 4:2:2 group spans two columns, and an even width is what makes the driver’s 2 * width row equal the 4 * ceil(width / 2) the schema derives, so the tensor addresses exactly the frame the descriptor describes.

A consumer whose declared rank, dtype, or placement differs from what the operator publishes will fail the port contract at graph authoring time. The rank and dtype are fixed by the payload above; the placement is the operator’s memory_kind, so the two must be chosen together.

Frame placement

The memory_kind constructor parameter selects where the frame lands. It is frozen into the port contract at setup(), so it is an authoring-time choice rather than something a running graph renegotiates, and a consumer’s expects_tensor must name the same placement.

The driver always delivers into a pageable mmap buffer, so every placement copies once out of it; the parameter chooses what that copy targets, not whether it happens. Use it to state where this operator’s own consumers want the data. Repairing a placement mismatch between two operators is an edge concern — see ConnectionOptions::memory_conversion — not a reason to change the producer.

kUnknown and kCudaManaged are rejected at construction.

kCudaDevice requires the application to name a GPU

Compilation will not choose a device for a device-resident pool; it rejects the plan with DEVICE_UNRESOLVED, because an implicitly chosen GPU is an implicitly chosen cross-device transfer later. The application supplies it, since which GPU to use is a deployment fact the operator does not own:

holoscan::CompileOptions options{};
options.deployment.bind_tensor_output_device(holoscan::TensorOutputDevicePlacement{
    .operator_path = "camera", .output_port = "frame", .device = holoscan::DeviceId{0}});
const holoscan::ExecutionPlan plan = holoscan::compile(graph, std::move(options));

The binding must be absent for kHost and process-local kPinnedHost, where supplying one is rejected as TENSOR_DEVICE_BINDING_UNEXPECTED. It is therefore not safe to set unconditionally — see --memory-kind handling in the reference application for the conditional form.

C++ Usage

#include <holoscan/core/compile.hpp>
#include <holoscan/core/graph.hpp>
#include <v4l2_capture_op/v4l2_capture_op.hpp>
namespace mm = holoscan::camera;
holoscan::Graph graph{"example"};
auto camera = graph.op<mm::V4l2CaptureOp>("camera", "/dev/video0", 640, 480, 30);

Capture counters

A C++ reference capture run ends with one summary line:

frames: received=N with-capture-time=N degraded=N corrupt=N unnamed-clock=N

Use these counters to check completion, timestamp coverage, and sample quality:

Reject samples carrying SampleFlags::kInvalid; inspect SampleFlags::kDegraded when deciding whether a partial frame or sequence discontinuity is acceptable. Check received against the requested frame count and with-capture-time against received to assess completion and timestamp coverage.

See the V4L2 benchmarks and qualification results for recorded platform behavior and the release notes for known issues.