Functional API¶
Quick start¶
Warning
This API is experimental and subject to change without notice!
Functional API is designed to simplify the usage of DALI operators in a psuedo-imperative way.
It exposes operators as functions, with ths same name as the operator class, but converted
to snake_case - for example ops.FileReader
will be exposed as fn.file_reader()
.
Example:
import nvidia.dali as dali
pipe = dali.pipeline.Pipeline(batch_size = 3, num_threads = 2, device_id = 0)
with pipe:
files, labels = dali.fn.file_reader(file_root = "./my_file_root")
images = dali.fn.image_decoder(files, device = "mixed")
images = dali.fn.rotate(images, angle = dali.fn.uniform(range=(-45,45)))
images = dali.fn.resize(images, resize_x = 300, resize_y = 300)
pipe.set_outputs(images, labels)
pipe.build()
outputs = pipe.run()
The use of functional API does not change other aspects of pipeline definition - the functions
still operate on and return pipeline.DataNode
objects.
Interoperability with operator objects¶
Functional API is, for the major part, only a wrapper around operator objects - as such, it is inherently compatible with the object-based API. The following example mixes the two, using object API to pre-configure a file reader and a resize operator:
pipe = dali.pipeline.Pipeline(batch_size = 3, num_threads = 2, device_id = 0)
reader = dali.ops.FileReader(file_root = ".")
resize = dali.ops.Resize(device = "gpu", resize_x = 300, resize_y = 300)
with pipe:
files, labels = reader()
images = dali.fn.image_decoder(files, device = "mixed")
images = dali.fn.rotate(images, angle = dali.fn.uniform(range=(-45,45)))
images = resize(images)
pipe.set_outputs(images, labels)
pipe.build()
outputs = pipe.run()
Functions¶
-
nvidia.dali.fn.
audio_decoder
(*inputs, **arguments)¶
-
nvidia.dali.fn.
bb_flip
(*inputs, **arguments)¶
-
nvidia.dali.fn.
bbox_paste
(*inputs, **arguments)¶
-
nvidia.dali.fn.
box_encoder
(*inputs, **arguments)¶
-
nvidia.dali.fn.
brightness
(*inputs, **arguments)¶
-
nvidia.dali.fn.
brightness_contrast
(*inputs, **arguments)¶
-
nvidia.dali.fn.
caffe2_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
caffe_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
cast
(*inputs, **arguments)¶
-
nvidia.dali.fn.
coco_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
coin_flip
(*inputs, **arguments)¶
-
nvidia.dali.fn.
color_space_conversion
(*inputs, **arguments)¶
-
nvidia.dali.fn.
color_twist
(*inputs, **arguments)¶
-
nvidia.dali.fn.
constant
(*inputs, **arguments)¶
-
nvidia.dali.fn.
contrast
(*inputs, **arguments)¶
-
nvidia.dali.fn.
coord_flip
(*inputs, **arguments)¶
-
nvidia.dali.fn.
copy
(*inputs, **arguments)¶
-
nvidia.dali.fn.
crop
(*inputs, **arguments)¶
-
nvidia.dali.fn.
crop_mirror_normalize
(*inputs, **arguments)¶
-
nvidia.dali.fn.
dl_tensor_python_function
(*inputs, **arguments)¶
-
nvidia.dali.fn.
dump_image
(*inputs, **arguments)¶
-
nvidia.dali.fn.
element_extract
(*inputs, **arguments)¶
-
nvidia.dali.fn.
erase
(*inputs, **arguments)¶
-
nvidia.dali.fn.
external_source
(source=None, num_outputs=None, *, cycle=None, name=None, device='cpu', layout=None, cuda_stream=None)¶ Creates a data node which is populated with data from a Python source. The data can be provided by the
source
function or iterable, or it can be provided bypipeline.feed_input(name, data, layout)
insidepipeline.iter_setup
.In the case of the GPU input, it is the user responsibility to modify the provided GPU memory content only using provided stream (DALI schedules a copy on it and all work is properly queued). If no stream is provided feeding input blocks until the provided memory is copied to the internal buffer
Note
To return a batch of copies of the same tensor, use
nvidia.dali.types.Constant()
, which is more performant.- Parameters
source (callable or iterable) – The source of the data. The source is polled for data (via a call
source()
ornext(source)
whenever the pipeline needs input for the next iteration. The source can supply one or more data batches, depending on the value ofnum_outputs
. Ifnum_outputs
is not set, thesource
is expected to return a single batch. If it’s specified, the data is expected to a be tuple or list where each element corresponds to respective return value of the external_source. If the source is a callable and has a positional argument, it is assumed to be the current iteration number and consecutive calls will besource(0)
,source(1)
, etc. If the source is a generator function, it is invoked and treated as an iterable - however, unlike a generator, it can be used withcycle
, in which case the function will be called again when the generator reaches end of iteration. In the case of the GPU input, it is the user responsibility to modify the provided GPU memory content only using provided stream (DALI schedules a copy on it and all work is properly queued). If no stream is provided, DALI will use a default, with best-effort approach at correctness (seecuda_stream
argument documentation for details).num_outputs (int, optional) – If specified, denotes the number of TensorLists produced by the source function
- Keyword Arguments
cycle (bool) – If
True
, the source will be wrapped. Otherwise, StopIteration will be raised when end of data is reached. This flag requires thatsource
is either a collection, i.e. an iterable object whereiter(source)
will return a fresh iterator on each call or a generator function. In the latter case, the generator function will be called again when more data is requested than was yielded by the function.name (str, optional) – The name of the data node - used when feeding the data in
iter_setup
; can be omitted if the data is provided bysource
.layout (layout str or list/tuple thereof) – If provided, sets the layout of the data. When
num_outputs > 1
, layout can be a list containing a distinct layout for each output. If the list has fewer elements thannum_outputs
, only the first outputs have the layout set, the reset have it cleared.cuda_stream (optional, cudaStream_t or an object convertible to cudaStream_t, e.g. cupy.cuda.Stream, torch.cuda.Stream) –
The CUDA stream, which is going to be used for copying data to GPU or from a GPU source. If not set, best effort will be taken to maintain correctness - i.e. if the data is provided as a tensor/array from a recognized library (CuPy, PyTorch), the library’s current stream is used. This should work in typical scenarios, but advanced use cases (and code using unsupported libraries) may still need to supply the stream handle explicitly.
- Special values:
0 - use default CUDA stream
-1 - use DALI’s internal stream
If internal stream is used, the call to
feed_input
will block until the copy to internal buffer is complete, since there’s no way to synchronize with this stream to prevent overwriting the array with new data in another stream.
-
nvidia.dali.fn.
fast_resize_crop_mirror
(*inputs, **arguments)¶
-
nvidia.dali.fn.
file_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
flip
(*inputs, **arguments)¶
-
nvidia.dali.fn.
gaussian_blur
(*inputs, **arguments)¶
-
nvidia.dali.fn.
hsv
(*inputs, **arguments)¶
-
nvidia.dali.fn.
hue
(*inputs, **arguments)¶
-
nvidia.dali.fn.
image_decoder
(*inputs, **arguments)¶
-
nvidia.dali.fn.
image_decoder_crop
(*inputs, **arguments)¶
-
nvidia.dali.fn.
image_decoder_random_crop
(*inputs, **arguments)¶
-
nvidia.dali.fn.
image_decoder_slice
(*inputs, **arguments)¶
-
nvidia.dali.fn.
jitter
(*inputs, **arguments)¶
-
nvidia.dali.fn.
lookup_table
(*inputs, **arguments)¶
-
nvidia.dali.fn.
mel_filter_bank
(*inputs, **arguments)¶
-
nvidia.dali.fn.
mfc
(*inputs, **arguments)¶
-
nvidia.dali.fn.
mxnet_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
nonsilent_region
(*inputs, **arguments)¶
-
nvidia.dali.fn.
normal_distribution
(*inputs, **arguments)¶
-
nvidia.dali.fn.
normalize
(*inputs, **arguments)¶
-
nvidia.dali.fn.
numpy_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
old_color_twist
(*inputs, **arguments)¶
-
nvidia.dali.fn.
one_hot
(*inputs, **arguments)¶
-
nvidia.dali.fn.
optical_flow
(*inputs, **arguments)¶
-
nvidia.dali.fn.
pad
(*inputs, **arguments)¶
-
nvidia.dali.fn.
paste
(*inputs, **arguments)¶
-
nvidia.dali.fn.
power_spectrum
(*inputs, **arguments)¶
-
nvidia.dali.fn.
preemphasis_filter
(*inputs, **arguments)¶
-
nvidia.dali.fn.
python_function
(*inputs, **arguments)¶
-
nvidia.dali.fn.
random_bbox_crop
(*inputs, **arguments)¶
-
nvidia.dali.fn.
random_resized_crop
(*inputs, **arguments)¶
-
nvidia.dali.fn.
reinterpret
(*inputs, **arguments)¶
-
nvidia.dali.fn.
reshape
(*inputs, **arguments)¶
-
nvidia.dali.fn.
resize
(*inputs, **arguments)¶
-
nvidia.dali.fn.
resize_crop_mirror
(*inputs, **arguments)¶
-
nvidia.dali.fn.
rotate
(*inputs, **arguments)¶
-
nvidia.dali.fn.
saturation
(*inputs, **arguments)¶
-
nvidia.dali.fn.
sequence_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
sequence_rearrange
(*inputs, **arguments)¶
-
nvidia.dali.fn.
shapes
(*inputs, **arguments)¶
-
nvidia.dali.fn.
slice
(*inputs, **arguments)¶
-
nvidia.dali.fn.
spectrogram
(*inputs, **arguments)¶
-
nvidia.dali.fn.
sphere
(*inputs, **arguments)¶
-
nvidia.dali.fn.
ssd_random_crop
(*inputs, **arguments)¶
-
nvidia.dali.fn.
tfrecord_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
to_decibels
(*inputs, **arguments)¶
-
nvidia.dali.fn.
transpose
(*inputs, **arguments)¶
-
nvidia.dali.fn.
uniform
(*inputs, **arguments)¶
-
nvidia.dali.fn.
video_reader
(*inputs, **arguments)¶
-
nvidia.dali.fn.
warp_affine
(*inputs, **arguments)¶
-
nvidia.dali.fn.
water
(*inputs, **arguments)¶