DALI Release 1.0.0

Using DALI 1.0.0

To upgrade to DALI 1.0.0 from an older version of DALI, follow the installation and usage information in the DALI User Guide.

Note: The internal DALI C++ API used for operator’s implementation, and the C++ API that enables using DALI as a library from native code, is not yet officially supported. Hence these APIs may change in the next release without advance notice.

Key Features and Enhancements

This DALI release includes the following key features and enhancements:

  • The API documentation has been improved:
  • New operators:
    • A GridMask GPU operator for GridMask data augmentation (#2652).
    • A RandomObjectBBox operator with caching to randomly select a bounding box (#2718, #2696, #2677, and #2657).
    • A MultiPaste operator, which is required to implement Mosaic augmentation (#2583).
  • External Source can now run the per-sample callbacks in parallel. (#2543)
  • Added pipeline_def decorator, which is an easier way to define a pipeline with the functional API (#2757 and #2629).
  • Moved all decoders to a dedicated Python module (#2741, #2743, and #2725).
  • Moved all readers to a dedicated Python module (#2720, #2721, #2717, #2715, and #2722).
  • Exposed the pipeline output names in the C API (#2665)
  • Introduced the following named Slice operator arguments (#2625):
    • start/rel_start
    • end/rel_end
    • shape/rel_shape
  • Enabled additional codecs and demuxers in FFmpeg (#2651).
  • Added an option to disable the first batch preparation during the iterator construction (#2664).

Fixed Issues

This DALI release includes the following fixes:

  • Fixed the JPEG 2000 ROI decoding (#2692).
  • Fixed the layout length check in Transpose (#2693).
  • Fixed the .gpu() usage detection and error for CPU-only pipelines (#2682).

Breaking Changes

  • There are no breaking changes in this DALI release.

Deprecated Features

The following features have been deprecated in this DALI release:
  • fn.audio_decoder/ops.AudioDecoder has been renamed to fn.decoders.audio/ops.decoders.Audio.
  • fn.image_decoder/ops.ImageDecoder has been renamed to fn.decoders.image/ops.decoders.Image.
  • fn.image_decoder_crop/ops.ImageDecoderCrop has been renamed to fn.decoders.image_crop/ops.decoders.ImageCrop.
  • fn.image_decoder_random_crop/ops.ImageDecoderRandomCrop has been renamed to fn.decoders.image_random_crop/ops.decoders.ImageRandomCrop.
  • fn.image_decoder_slice/ops.ImageDecoderSlice has been renamed to fn.decoders.image_slice/ops.decoders.ImageSlice.
  • fn.caffe2_reader/ops.Caffe2Reader has been renamed to fn.readers.caffe2/ops.readers.Caffe2.
  • fn.caffe_reader/ops.CaffeReader has been renamed to fn.readers.caffe/ops.readers.Caffe.
  • fn.coco_reader/ops.CocoReader has been renamed to fn.readers.coco/ops.readers.Coco.
  • fn.file_reader/ops.FileReader has been renamed to fn.readers.file/ops.readers.File.
  • fn.mxnet_reader/ops.MXNetReader has been renamed to fn.readers.mxnet/ops.readers.MXNet.
  • fn.nemo_asr_reader/ops.NemoAsrReader has been renamed to fn.readers.nemo_asr/ops.readers.NemoAsr.
  • fn.numpy_reader/ops.NumpyReader has been renamed to fn.readers.numpy/ops.readers.Numpy.
  • fn.sequence_reader/ops.SequenceReader has been renamed to fn.readers.sequence/ops.readers.Sequence.
  • fn.tfrecord_reader/ops.TFRecordReader has been renamed to fn.readers.tfrecord/ops.readers.TFRecord.
  • fn.video_reader/ops.VideoReader has been renamed to fn.readers.video/ops.readers.Video.
  • fn.video_reader_resize/ops.VideoReaderResize has been renamed to fn.readers.video_resize/ops.readers.VideoResize.
  • The split implementation of decoders.image was removed, and the split_stages and use_chunk_allocator arguments have been deprecated.

Known Issues

This DALI release includes the following known issues:

  • The video loader operator requires that the key frames occur, at a minimum, every 10 to 15 frames of the video stream.

    If the key frames occur at a frequency that is less than 10-15 frames, the returned frames might be out of sync.

  • The DALI TensorFlow plugin might not be compatible with TensorFlow versions 1.15.0 and later.

    To use DALI with the TensorFlow version that does not have a prebuilt plugin binary that is shipped with DALI, make sure that the compiler that is used to build TensorFlow exists on the system during the plugin installation. (Depending on the particular version, you can use GCC 4.8.4, GCC 4.8.5, or GCC 5.4.)

  • Due to some known issues with meltdown/spectra mitigations and DALI, DALI shows the best performance when running in Docker with escalated privileges, for example:
    • privileged=yes in Extra Settings for AWS data points
    • --privileged or --security-opt seccomp=unconfined for bare Docker