{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# DALI Expressions and Arithmetic Operators\n", "\n", "This example shows you how to use binary arithmetic operators in the DALI Pipeline that allow for element-wise operations on tensors in a pipeline. We will provide information about the available operators and examples of using constant and scalar inputs.\n", "\n", "## Supported Operators\n", "\n", "DALI currently supports the following operators: \n", "\n", " - Unary arithmetic operators: `+`, `-`;\n", "\n", " - Binary arithmetic operators: `+`, `-`, `*`, `/`, and `//`;\n", "\n", " - Comparison operators: `==`, `!=`, `<`, `<=`, `>`, `>=`;\n", "\n", " - Bitwise binary operators: `&`, `|`, `^`.\n", " \n", " \n", "DALI also supports set of mathematical functions exposed in the `nvidia.dali.math` module:\n", "\n", " - `abs`, `fabs`, `floor`, `ceil`, `pow`, `fpow`, `min`, `max`, `clamp`;\n", "\n", " - Exponents and logarithms: `sqrt`, `rsqrt`, `cbrt`, `exp`, `log`, `log2`, `log10`;\n", "\n", " - Trigonometric Functions: `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `atan2`;\n", "\n", " - Hyperbolic Functions: `sinh`, `cosh`, `tanh`, `asinh`, `acosh`, `atanh`;\n", " \n", "Binary operators can be used as an operation between two tensors, between a tensor and a scalar or a tensor and a constant. By tensor, we consider the output of DALI operators (regular or other arithmetic operators). Unary operators work only with tensor inputs.\n", "\n", "In this section we focus on binary arithmetic operators, Tensor, Constant and Scalar operands. The detailed type promotion rules for comparison and bitwise operators are covered in the **Supported operations** section and other examples." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Prepare the Test Pipeline\n", "\n", "1. Prepare the helper code, so we can easily manipulate the types and values that will appear as tensors in the DALI pipeline.\n", "\n", "2. We will be using numpy as source for the custom provided data, so we need to import several things from DALI needed to create the Pipeline and use the ExternalSource operator. " ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "from nvidia.dali.pipeline import pipeline_def\n", "import nvidia.dali.fn as fn\n", "import nvidia.dali.types as types\n", "from nvidia.dali.types import Constant" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Defining the Data\n", "\n", "To define the data, because there are binary operators, two inputs are required. \n", "We will create a simple helper function that returns two batches of hardcoded data, that are stored as `np.int32`. In an actual scenario the data that is processed by the DALI arithmetic operators would be tensors produced by another operator that contains some images, video sequences or other data.\n", "\n", "You can experiment by changing those values or adjusting the `get_data()` function to use different input data. \n", "\n", "**Note**: The shapes of both inputs need to either match exactly or be compatible according to the broadcasting rules - the operations are performed element-wise. " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "left_magic_values = [[[42, 7, 0], [0, 0, 0]], [[5, 10, 15], [10, 100, 1000]]]\n", "\n", "right_magic_values = [[[3, 3, 3], [1, 3, 5]], [[1, 5, 5], [1, 1, 1]]]\n", "\n", "batch_size = len(left_magic_values)\n", "\n", "\n", "def convert_batch(batch):\n", " return [np.int32(tensor) for tensor in batch]\n", "\n", "\n", "def get_data():\n", " return (convert_batch(left_magic_values), convert_batch(right_magic_values))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Operating on Tensors\n", "\n", "### Defining the Pipeline\n", "\n", "1. To define the pipeline, the data will be obtained from the `get_data` function and made available to the pipeline through `ExternalSource`.\n", "\n", "**Note**: You do not need to instantiate any additional operators, we can use regular Python arithmetic expressions on the results of other operators.\n", "\n", "2. You can manipulate the source data by adding, multiplying and dividing it." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "@pipeline_def(batch_size=batch_size, num_threads=4, device_id=0)\n", "def pipeline():\n", " l, r = fn.external_source(source=get_data, num_outputs=2, dtype=types.INT32)\n", " sum_result = l + r\n", " mul_result = l * r\n", " div_result = l // r\n", " return l, r, sum_result, mul_result, div_result" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Running the Pipeline\n", "\n", "3. Build and run our pipeline" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "pipe = pipeline()\n", "pipe.build()\n", "out = pipe.run()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "4. Display the results:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[[ 42 7 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 10 15]\n", " [ 10 100 1000]]]\n", "+\n", "[[[3 3 3]\n", " [1 3 5]]\n", "\n", " [[1 5 5]\n", " [1 1 1]]]\n", "=\n", "[[[ 45 10 3]\n", " [ 1 3 5]]\n", "\n", " [[ 6 15 20]\n", " [ 11 101 1001]]]\n", "\n", "\n", "[[[ 42 7 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 10 15]\n", " [ 10 100 1000]]]\n", "*\n", "[[[3 3 3]\n", " [1 3 5]]\n", "\n", " [[1 5 5]\n", " [1 1 1]]]\n", "=\n", "[[[ 126 21 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 50 75]\n", " [ 10 100 1000]]]\n", "\n", "\n", "[[[ 42 7 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 10 15]\n", " [ 10 100 1000]]]\n", "//\n", "[[[3 3 3]\n", " [1 3 5]]\n", "\n", " [[1 5 5]\n", " [1 1 1]]]\n", "=\n", "[[[ 14 2 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 2 3]\n", " [ 10 100 1000]]]\n", "\n", "\n" ] } ], "source": [ "def examine_output(pipe_out):\n", " l = pipe_out[0].as_array()\n", " r = pipe_out[1].as_array()\n", " sum_out = pipe_out[2].as_array()\n", " mul_out = pipe_out[3].as_array()\n", " div_out = pipe_out[4].as_array()\n", " print(\"{}\\n+\\n{}\\n=\\n{}\\n\\n\".format(l, r, sum_out))\n", " print(\"{}\\n*\\n{}\\n=\\n{}\\n\\n\".format(l, r, mul_out))\n", " print(\"{}\\n//\\n{}\\n=\\n{}\\n\\n\".format(l, r, div_out))\n", "\n", "\n", "examine_output(out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The resulting tensors are obtained by applying the arithmetic operation between corresponding elements of its inputs.\n", "\n", "With an exception for scalar tensor inputs that we will describe in the next section, the shapes of the arguments should be compatible - either match exactly or be broadcastable, otherwise you will get an error. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Constant and Scalar Operands\n", "Until now we considered only tensor inputs of matching shapes for inputs of arithmetic operators. DALI allows one of the operands to be a constant or a batch of scalars, and such operands can appear on both sides of binary expressions.\n", "\n", "## Constants\n", "The constant operand for arithmetic operator can be one of the following options: \n", "\n", " - Values of Python's `int` and `float` types that are used directly.\n", " \n", " - Values that are wrapped in `nvidia.dali.types.Constant`.\n", " \n", "The operation between the tensor and the constant results in the constant that is broadcast to all tensor elements.\n", "\n", "**Note**: Currently, the values of the integral constants are passed internally to DALI as int32 and the values of floating point constants are passed to DALI as float32.\n", "\n", "With regard to type promotion, the Python `int` values will be treated as `int32` and the `float` as `float32`.\n", "\n", "The DALI `Constant` can be used to indicate other types. It accepts `DALIDataType` enum values as second argument and has convenience member functions such as `.uint8()` or `.float32()` that can be used for conversions.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Using the Constants\n", "\n", "1. Adjust the Pipeline to utilize constants. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "@pipeline_def(batch_size=batch_size, num_threads=4, device_id=0)\n", "def pipeline_2():\n", " l, r = fn.external_source(source=get_data, num_outputs=2, dtype=types.INT32)\n", " add_200 = l + 200\n", " mul_075 = l * 0.75\n", " sub_15 = Constant(15).float32() - r\n", " return l, r, add_200, mul_075, sub_15" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "pipe = pipeline_2()\n", "pipe.build()\n", "out = pipe.run()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "2. Display the results:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[[ 42 7 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 10 15]\n", " [ 10 100 1000]]]\n", "+ 200 =\n", "[[[ 242 207 200]\n", " [ 200 200 200]]\n", "\n", " [[ 205 210 215]\n", " [ 210 300 1200]]]\n", "\n", "\n", "[[[ 42 7 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 10 15]\n", " [ 10 100 1000]]]\n", "* 0.75 =\n", "[[[ 31.5 5.25 0. ]\n", " [ 0. 0. 0. ]]\n", "\n", " [[ 3.75 7.5 11.25]\n", " [ 7.5 75. 750. ]]]\n", "\n", "\n", "15 -\n", "[[[3 3 3]\n", " [1 3 5]]\n", "\n", " [[1 5 5]\n", " [1 1 1]]]\n", "=\n", "[[[12. 12. 12.]\n", " [14. 12. 10.]]\n", "\n", " [[14. 10. 10.]\n", " [14. 14. 14.]]]\n", "\n", "\n" ] } ], "source": [ "def examine_output(pipe_out):\n", " l = pipe_out[0].as_array()\n", " r = pipe_out[1].as_array()\n", " add_200 = pipe_out[2].as_array()\n", " mul_075 = pipe_out[3].as_array()\n", " sub_15 = pipe_out[4].as_array()\n", " print(\"{}\\n+ 200 =\\n{}\\n\\n\".format(l, add_200))\n", " print(\"{}\\n* 0.75 =\\n{}\\n\\n\".format(l, mul_075))\n", " print(\"15 -\\n{}\\n=\\n{}\\n\\n\".format(r, sub_15))\n", "\n", "\n", "examine_output(out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The constant value is used with all elements of all tensors in the batch." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Broadcasting\n", "The term “broadcasting” refers to how tensors with different shapes are treated in mathematical expressions. A value from a smaller tensor is “broadcast” so it contributes to multiple output values. At its simplest, a scalar value is broadcast to all output values. The broadcasting between two batches can be seen as generalization of this concept.\n", "\n", "In more complex cases, the values can be broadcast along some dimensions if one of the operands has size `1` and the other is larger. \n", "For example tensor of shape `(1, 3)` can be broadcast along the outermost dimension when added to a tensor with `(2, 3)` shape. Let's add two such constants and see the result." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "@pipeline_def(batch_size=1, num_threads=4, device_id=0)\n", "def pipeline_3():\n", " left = Constant(np.float32([[1.0, 2.0, 3.0]]))\n", " right = Constant(np.float32([[-5.0, -6.0, -7.0], [10.0, 20.0, 30.0]]))\n", " return left + right" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TensorListCPU(\n", " [[[-4. -4. -4.]\n", " [11. 22. 33.]]],\n", " dtype=DALIDataType.FLOAT,\n", " num_samples=1,\n", " shape=[(2, 3)])\n" ] } ], "source": [ "pipe = pipeline_3()\n", "pipe.build()\n", "(out,) = pipe.run()\n", "print(out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Examining the output, we can see that the `left` tensor was added to both rows of the `right` tensor. \n", "\n", "In fact, we don't have to create a tensor with the leading dimensions equal to `1` - all the missing dimensions would be padded by `1`. For example, broadcasting operation between tensors of shapes `(3)` and `(800, 600, 3)` is equivalent to broadcasting between `(1, 1, 3)` and `(800, 600, 3)`.\n", "\n", "Keep in mind, that every sample is broadcast individually. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Broadcasting scalars\n", "\n", "Batch of scalars (0D tensors) can be broadcast against any other batch. \n", "\n", "1. Use an `ExternalSource` to generate a sequence of numbers which will be added to the tensor operands." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "@pipeline_def(batch_size=batch_size, num_threads=4, device_id=0)\n", "def pipeline_4():\n", " tensors = fn.external_source(lambda: get_data()[0], dtype=types.INT32)\n", " scalars = fn.external_source(lambda: np.arange(1, batch_size + 1), dtype=types.INT64)\n", " return tensors, scalars, tensors + scalars" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "2. Build and run the Pipeline. " ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "pipe = pipeline_4()\n", "pipe.build()\n", "out = pipe.run()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[[[ 42 7 0]\n", " [ 0 0 0]]\n", "\n", " [[ 5 10 15]\n", " [ 10 100 1000]]]\n", "+\n", "[1 2]\n", "=\n", "[[[ 43 8 1]\n", " [ 1 1 1]]\n", "\n", " [[ 7 12 17]\n", " [ 12 102 1002]]]\n" ] } ], "source": [ "def examine_output(pipe_out):\n", " t = pipe_out[0].as_array()\n", " scalar = pipe_out[1].as_array()\n", " result = pipe_out[2].as_array()\n", " print(\"{}\\n+\\n{}\\n=\\n{}\".format(t, scalar, result))\n", "\n", "\n", "examine_output(out)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The first scalar in the batch (1) is added to all elements in the first tensor, and the second scalar (2) is added to the second tensor." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" } }, "nbformat": 4, "nbformat_minor": 2 }