nvidia.dali.fn.optical_flow

nvidia.dali.fn.optical_flow(*inputs, **kwargs)

Calculates the optical flow between images in the input.

The main input for this operator is a sequence of frames. Optionally, the operator can be provided with external hints for the optical flow calculation. The output format of this operator matches the output format of the optical flow driver API. Refer to https://developer.nvidia.com/opticalflow-sdk for more information about the Turing and Ampere optical flow hardware that is used by DALI.

This operator allows sequence inputs.

Supported backends

  • ‘gpu’

Parameters

  • input0 (TensorList ('FHWC')) – Input to the operator.

  • input1 (TensorList ('FHWC'), optional) – Input to the operator.

Keyword Arguments

  • bytes_per_sample_hint (int or list of int, optional, default = [0]) –

    Output size hint, in bytes per sample.

    If specified, the operator’s outputs residing in GPU or page-locked host memory will be preallocated to accommodate a batch of samples of this size.

  • enable_external_hints (bool, optional, default = False) –

    Enables or disables the external hints for optical flow calculations.

    External hints are analogous to temporal hints, but the only difference is that external hints come from an external source. When this option is enabled, the operator requires two inputs.

  • enable_temporal_hints (bool, optional, default = False) –

    Enables or disables temporal hints for sequences that are longer than two images.

    The hints are used to improve the quality of the output motion field as well as to speed up the calculations. The hints are especially useful in presence of large displacements or periodic patterns which might confuse the optical flow algorithms. )

  • hint_grid (int, optional, default = 4) –

    Sets the grid size for the hint vector field.

    The hints are used to improve the quality of the output motion field as well as to speed up the calculations. The grid resolution could be set to a different value than the output.

    Note

    Currently, only a 1, 2, 4 and 8 are supported for Ampere and 4 for Turing.

  • image_type (nvidia.dali.types.DALIImageType, optional, default = DALIImageType.RGB) – Input color space (RGB, BGR or GRAY).

  • output_grid (int, optional, default = 4) –

    Sets the grid size for the output vector field.

    This operator produces the motion vector field at a coarser resolution than the input pixels. This parameter specifies the size of the pixel grid cell corresponding to one motion vector. For example, a value of 4 will produce one motion vector for each 4x4 pixel block.

    Note

    Currently, only a 1, 2 and 4 are supported for Ampere and 4 for Turing.

  • preserve (bool, optional, default = False) – Prevents the operator from being removed from the graph even if its outputs are not used.

  • preset (float, optional, default = 0.0) –

    Speed and quality level of the optical flow calculation.

    Allowed values are:

    • 0.0 is the lowest speed and the best quality.

    • 0.5 is the medium speed and quality.

    • 1.0 is the fastest speed and the lowest quality.

    The lower the speed, the more additional pre- and postprocessing is used to enhance the quality of the optical flow result.

  • seed (int, optional, default = -1) –

    Random seed.

    If not provided, it will be populated based on the global seed of the pipeline.

  • output_format (int) –

    Warning

    The argument output_format is a deprecated alias for output_grid. Use output_grid instead.