Augmentation operations

In terms of the automatic augmentations, the augmentation is image processing function that meets following requirements:

  1. Its first argument is the input batch for the processing

  2. The second argument is the parameter controlling the operation (for example angle of rotation).

  3. It can take additional keyword arguments.

  4. It is implemented in terms of DALI operators.

  5. It is decorated with @augmentation

Here is an example of defining a simplified rotate augmentation:

from nvidia.dali.auto_aug.core import augmentation
from nvidia.dali import fn

@augmentation(mag_range=(0, 30), randomly_negate=True)
def rotate_aug(data, angle, fill_value=128, rotate_keep_size=True):
   return fn.rotate(data, angle=angle, fill_value=fill_value, keep_size=True)

Based on the existing augmentation, a new one, with adjusted parameters, can be created:

rotate_aug_60 = rotate_aug.augmentation(mag_range=(0, 60), randomly_negate=False)

To learn more how to build a policy using augmentations listed here, check the documentation for specific automatic augmentation scheme: AutoAugment, RandAugment, or TrivialAugment.

Decorator

@nvidia.dali.auto_aug.core.augmentation(function=None, *, mag_range=None, randomly_negate=None, mag_to_param=None, param_device=None, name=None, augmentation_cls=None)

A decorator turning transformations implemented with DALI into augmentations that can be used by automatic augmentations (e.g. AutoAugment, RandAugment, TrivialAugment).

The function must accept at least two args: a sample and a parameter. The decorator handles computation of the parameter. Instead of the parameter, the decorated augmentation accepts magnitude bin and the total number of bins. Then, the bin is used to compute the parameter as if by calling mag_to_param(magnitudes[magnitude_bin] * ((-1) ** random_sign)), where magnitudes=linspace(mag_range[0], mag_range[1], num_magnitude_bins).

Parameters:
  • function (callable) – A function that accepts at least two positional args: a batch (represented as DataNode) to be processed, and a parameter of the transformation. The function must use DALI operators to process the batch and return a single output of such processing.

  • mag_range ((number, number) or np.ndarray) – Specifies the range of applicable magnitudes for the operation. If the tuple is provided, the magnitudes will be computed as linspace(mag_range[0], mag_range[1], num_magnitude_bins). If the np.ndarray is provided, it will be used directly instead of the linspace. If no mag_range is specified, the parameter passed to the function will be None.

  • randomly_negate (bool) – If True, the magnitude from the mag_range will be randomly negated for every sample.

  • mag_to_param (callable) – A callback that transforms the magnitude into a parameter. The parameter will be passed to the decorated operation instead of the plain magnitude. This way, the parameters for the range of magnitudes can be computed once in advance and stored as a Constant node. Note, the callback must return numpy arrays or data directly convertible to numpy arrays (in particular, no pipelined DALI operators can be used in the callback). The output type and dimensionality must be consistent and not depend on the magnitude.

  • param_device (str) – A “cpu”, “gpu”, or “auto”; defaults to “cpu”. Describes where to store the precomputed parameters (i.e. the mag_to_param outputs). If “auto” is specified, the CPU or GPU backend will be selected to match the sample’s backend.

  • name (str) – Name of the augmentation. By default, the name of the decorated function is used.

Returns:

The operation wrapped with the Augmentation class so that it can be used with the auto_aug transforms.

Return type:

Augmentation

class nvidia.dali.auto_aug.core._augmentation.Augmentation

The result of decorating a function with @augmentation is an instance of class Augmentation(). The class should not be instantiated directly, it needs to be created with the decorator.

Once obtained, those objects become callables that can be used to specify a policy for AutoAugment, RandAugment or TrivialAugment.

def augmentation(self, mag_range, randomly_negate, mag_to_param, param_device, name) -> Augmentation

You can call this method to create new Augmentation() instance based on an existing one, with the parameters adjusted. All parameters are optional - those that were specified replace the ones that were previously passed to @augmentation.

Parameters:

Augmentations

Here is a list of callable Augmentation() instances defined by DALI. Note that the mag_to_param, param_device and name parameters were ommitted from the @augmentation decorator listing for simplicity.

To adjust the range of parameter, use the augmentation method on the existing Augmentation() instance listed below, for example:

# Create a steeper sheer operation based on existing one
steep_shear_x = shear_x.augmentation(mag_range=(0, 0.5), name="steep_shear_x")
nvidia.dali.auto_aug.augmentations.shear_x(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies nvidia.dali.fn.transforms.shear() with shear_x factor using nvidia.dali.fn.warp_affine().

@augmentation(mag_range=(0, 0.3), randomly_negate=True, ...)
def shear_x(data, shear, fill_value=128, interp_type=None)
nvidia.dali.auto_aug.augmentations.shear_y(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies nvidia.dali.fn.transforms.shear() with shear_y factor using nvidia.dali.fn.warp_affine().

@augmentation(mag_range=(0, 0.3), randomly_negate=True, ...)
def shear_y(data, shear, fill_value=128, interp_type=None)
nvidia.dali.auto_aug.augmentations.translate_x(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies nvidia.dali.fn.transforms.translation() with shape-relative offset in x-axis using nvidia.dali.fn.warp_affine().

@augmentation(mag_range=(0., 1.), randomly_negate=True, ...)
def translate_x(data, rel_offset, shape, fill_value=128, interp_type=None)
nvidia.dali.auto_aug.augmentations.translate_x_no_shape(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies nvidia.dali.fn.transforms.translation() with absolute offset in x-axis using nvidia.dali.fn.warp_affine().

@augmentation(mag_range=(0, 250), randomly_negate=True, ...)
def translate_x_no_shape(data, offset, fill_value=128, interp_type=None)
nvidia.dali.auto_aug.augmentations.translate_y(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies nvidia.dali.fn.transforms.translation() with shape-relative offset in y-axis using nvidia.dali.fn.warp_affine().

@augmentation(mag_range=(0., 1.), randomly_negate=True, ...)
def translate_y(data, rel_offset, shape, fill_value=128, interp_type=None)
nvidia.dali.auto_aug.augmentations.translate_y_no_shape(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies nvidia.dali.fn.transforms.translation() with absolute offset in y-axis using nvidia.dali.fn.warp_affine().

@augmentation(mag_range=(0, 250), randomly_negate=True, ...)
def translate_y_no_shape(data, offset, fill_value=128, interp_type=None)
nvidia.dali.auto_aug.augmentations.rotate(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Rotates the image using nvidia.dali.fn.rotate().

@augmentation(mag_range=(0, 30), randomly_negate=True)
def rotate(data, angle, fill_value=128, interp_type=None, rotate_keep_size=True)
nvidia.dali.auto_aug.augmentations.brightness(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Adjusts the brightness with nvidia.dali.fn.brightness(). The magnitude is mapped to a [0, 2] parameter range.

@augmentation(mag_range=(0, 0.9), randomly_negate=True, ...)
def brightness(data, parameter)
nvidia.dali.auto_aug.augmentations.contrast(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Adjusts the contrasts using a channel-weighted mean as a contrast center. The magnitude is mapped to a [0, 2] parameter range.

@augmentation(mag_range=(0, 0.9), randomly_negate=True, ...)
def contrast(data, parameter)
nvidia.dali.auto_aug.augmentations.color(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Adjusts the color with nvidia.dali.fn.saturation(). The magnitude is mapped to a [0, 2] parameter range.

@augmentation(mag_range=(0, 0.9), randomly_negate=True, ...)
def color(data, parameter)
nvidia.dali.auto_aug.augmentations.sharpness(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

The outputs correspond to PIL’s ImageEnhance.Sharpness.

@augmentation(mag_range=(0, 0.9), randomly_negate=True, ...)
def sharpness(data, kernel)
nvidia.dali.auto_aug.augmentations.posterize(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Posterizes the image by masking out the lower input bits.

@augmentation(mag_range=(0, 4), ...)
def posterize(data, mask)
nvidia.dali.auto_aug.augmentations.solarize(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Inverts the pixels that lie below a threshold.

@augmentation(mag_range=(256, 0), ...)
def solarize(data, threshold)
nvidia.dali.auto_aug.augmentations.solarize_add(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies the shift to the pixels of value lower than 128.

@augmentation(mag_range=(0, 110), ...)
def solarize_add(data, shift)
nvidia.dali.auto_aug.augmentations.invert(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Inverts the image.

@augmentation
def invert(data, _)
nvidia.dali.auto_aug.augmentations.equalize(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies histogram equalization using nvidia.dali.fn.experimental.equalize.

@augmentation
def equalize(data, _)
    """
    DALI's equalize follows OpenCV's histogram equalization.
    The PIL uses slightly different formula when transforming histogram's
    cumulative sum into lookup table.
nvidia.dali.auto_aug.augmentations.auto_contrast(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Applies automatic contrast adjustment.

@augmentation
def auto_contrast(data, _)
nvidia.dali.auto_aug.augmentations.identity(data, *, magnitude_bin=None, num_magnitude_bins=None, **kwargs)

Identity operation - no processing is applied.

@augmentation
def identity(data, _)