nvidia.dali.fn.optical_flow

nvidia.dali.fn.optical_flow(__input_0, __input_1=None, /, *, bytes_per_sample_hint=[0], enable_external_hints=False, enable_temporal_hints=False, hint_grid=4, image_type=DALIImageType.RGB, output_grid=4, preserve=False, preset=0.0, seed=-1, device=None, name=None)

Calculates the optical flow between images in the input.

The main input for this operator is a sequence of frames. Optionally, the operator can be provided with external hints for the optical flow calculation. The output format of this operator matches the output format of the optical flow driver API. Refer to https://developer.nvidia.com/opticalflow-sdk for more information about the Turing, Ampere and Hopper optical flow hardware that is used by DALI.

Note

The calculated optical flow is always with respect to the resolution of the input image, however the output optical flow image can be a lower resolution, dictated by output_grid. If instead you would like the optical flow vectors be consistent with the resolution of the output of this operator, you must divide the output vector field by output_grid.

This operator allows sequence inputs.

Supported backends

  • ‘gpu’

Parameters:

  • __input_0 (TensorList ('FHWC')) – Input to the operator.

  • __input_1 (TensorList ('FHWC'), optional) – Input to the operator.

Keyword Arguments:

  • bytes_per_sample_hint (int or list of int, optional, default = [0]) –

    Output size hint, in bytes per sample.

    If specified, the operator’s outputs residing in GPU or page-locked host memory will be preallocated to accommodate a batch of samples of this size.

  • enable_external_hints (bool, optional, default = False) –

    Enables or disables the external hints for optical flow calculations.

    External hints are analogous to temporal hints, but the only difference is that external hints come from an external source. When this option is enabled, the operator requires two inputs.

  • enable_temporal_hints (bool, optional, default = False) –

    Enables or disables temporal hints for sequences that are longer than two images.

    The hints are used to improve the quality of the output motion field as well as to speed up the calculations. The hints are especially useful in presence of large displacements or periodic patterns which might confuse the optical flow algorithms. )

  • hint_grid (int, optional, default = 4) –

    Sets the grid size for the hint vector field.

    The hints are used to improve the quality of the output motion field as well as to speed up the calculations. The grid resolution could be set to a different value than the output.

    Note

    Currently, only a 1, 2, 4 and 8 are supported for Ampere and 4 for Turing.

  • image_type (nvidia.dali.types.DALIImageType, optional, default = DALIImageType.RGB) – Input color space (RGB, BGR or GRAY).

  • output_grid (int, optional, default = 4) –

    Sets the grid size for the output vector field.

    This operator produces the motion vector field at a coarser resolution than the input pixels. This parameter specifies the size of the pixel grid cell corresponding to one motion vector. For example, a value of 4 will produce one motion vector for each 4x4 pixel block. Hence, to use optical flow with an output_grid of 4 to resample a full resolution image, the flow field is upsampled without scaling the vector quantities.

    Note

    Currently, only a 1, 2 and 4 are supported for Ampere and 4 for Turing.

  • preserve (bool, optional, default = False) – Prevents the operator from being removed from the graph even if its outputs are not used.

  • preset (float, optional, default = 0.0) –

    Speed and quality level of the optical flow calculation.

    Allowed values are:

    • 0.0 is the lowest speed and the best quality.

    • 0.5 is the medium speed and quality.

    • 1.0 is the fastest speed and the lowest quality.

    The lower the speed, the more additional pre- and postprocessing is used to enhance the quality of the optical flow result.

  • seed (int, optional, default = -1) –

    Random seed.

    If not provided, it will be populated based on the global seed of the pipeline.

  • output_format (int) –

    Warning

    The argument output_format is a deprecated alias for output_grid. Use output_grid instead.

See also