{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DALI expressions and arithmetic operators\n",
    "\n",
    "In this example, we will show simple examples how to use binary arithmetic operators in DALI Pipeline that allow for element-wise operations on tensors inside a pipeline. We will show available operators and examples of using constant and scalar inputs.\n",
    "\n",
    "## Supported operators\n",
    "\n",
    "DALI currently supports unary arithmetic operators: `+`, `-`; binary arithmetic operators: `+`, `-`, `*`, `/`, and `//`; comparison operators: `==`, `!=`, `<`, `<=`, `>`, `>=`; and bitwise binary operators: `&`, `|`, `^`.  Binary operators can be used as an operation between two tensors, between a tensor and a scalar or a tensor and a constant. By tensor we consider the output of DALI operators (either regular ones or other arithmetic operators). Unary operators work only with Tensor inputs.\n",
    "\n",
    "We will focus on binary arithmetic operators, Tensor, Constatn and Scalar operands. The detailed type promotion rules for comparison and bitwise operators are covered in the **Supported operations** section of documentation as well as other examplesarithmetic."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the test pipeline\n",
    "\n",
    "First, we will prepare the helper code, so we can easily manipulate the types and values that will appear as tensors in the DALI pipeline.\n",
    "\n",
    "We use `from __future__ import division` to allow `/` and `//` as true division and floor division operators.\n",
    "We will be using numpy as source for the custom provided data and we also need to import several things from DALI needed to create Pipeline and use ExternalSource Operator. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import division\n",
    "import numpy as np\n",
    "from nvidia.dali.pipeline import Pipeline\n",
    "import nvidia.dali.ops as ops            \n",
    "import nvidia.dali.types as types\n",
    "from nvidia.dali.types import Constant"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining the data\n",
    "\n",
    "As we are dealing with binary operators, we need two inputs. \n",
    "We will create a simple helper function that returns two batches of hardcoded data, stored as `np.int32`. In an actual scenario the data processed by DALI arithmetic operators would be tensors produced by other Operator containing some images, video sequences or other data.\n",
    "\n",
    "You can experiment by changing those values or adjusting the `get_data()` function to use different input data. Keep in mind that shapes of both inputs need to match as those will be element-wise operations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "left_magic_values = [\n",
    "    [[42, 7, 0], [0, 0, 0]], \n",
    "    [[5, 10, 15], [10, 100, 1000]]\n",
    "]\n",
    "\n",
    "right_magic_values = [\n",
    "    [[3, 3, 3], [1, 3, 5]], \n",
    "    [[1, 5, 5], [1, 1, 1]]\n",
    "]\n",
    "\n",
    "batch_size = len(left_magic_values)\n",
    "\n",
    "def convert_batch(batch):\n",
    "    return [np.int32(tensor) for tensor in batch]\n",
    "\n",
    "def get_data():\n",
    "    return (convert_batch(left_magic_values), convert_batch(right_magic_values))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Operating on tensors\n",
    "\n",
    "### Defining the pipeline\n",
    "\n",
    "The next step is to define the Pipeline. We override `Pipeline.iter_setup`, a method called by the pipeline before every `Pipeline.run`. It is meant to feed the data into `ExternalSource()` operators indicated by `self.left` and `self.right`.\n",
    "The data will be obtained from `get_data` function to which we pass the left and right types. We are using input of type `np.int32` for now.\n",
    "\n",
    "Note, that we do not need to instantiate any additional operators, we can use regular Python arithmetic expressions on the results of other operators in the `define_graph` step.\n",
    "\n",
    "Let's manipulate the source data by adding, multiplying and dividing it. `define_graph` will return both our data inputs and the result of applying arithmetic operations to them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    " class ArithmeticPipeline(Pipeline):                   \n",
    "    def __init__(self, batch_size, num_threads, device_id):\n",
    "        super(ArithmeticPipeline, self).__init__(batch_size, num_threads, device_id, seed=12)\n",
    "        self.left_source = ops.ExternalSource()\n",
    "        self.right_source = ops.ExternalSource()\n",
    "\n",
    "    def define_graph(self):                                                                \n",
    "        self.left = self.left_source()\n",
    "        self.right = self.right_source()\n",
    "        sum_result = self.left + self.right\n",
    "        mul_result = self.left * self.right\n",
    "        div_result = self.left // self.right\n",
    "        return self.left, self.right, sum_result, mul_result, div_result\n",
    "\n",
    "    def iter_setup(self):\n",
    "        # Get the data batches with selected types\n",
    "        (l, r) = get_data()\n",
    "        # Feed it to external source\n",
    "        self.feed_input(self.left, l)\n",
    "        self.feed_input(self.right, r)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running the pipeline\n",
    "\n",
    "Lets build and run our pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = ArithmeticPipeline(batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "pipe.build()\n",
    "out = pipe.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to display the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[  42    7    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5   10   15]\n",
      "  [  10  100 1000]]]\n",
      "+\n",
      "[[[3 3 3]\n",
      "  [1 3 5]]\n",
      "\n",
      " [[1 5 5]\n",
      "  [1 1 1]]]\n",
      "=\n",
      "[[[  45   10    3]\n",
      "  [   1    3    5]]\n",
      "\n",
      " [[   6   15   20]\n",
      "  [  11  101 1001]]]\n",
      "\n",
      "\n",
      "[[[  42    7    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5   10   15]\n",
      "  [  10  100 1000]]]\n",
      "*\n",
      "[[[3 3 3]\n",
      "  [1 3 5]]\n",
      "\n",
      " [[1 5 5]\n",
      "  [1 1 1]]]\n",
      "=\n",
      "[[[ 126   21    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5   50   75]\n",
      "  [  10  100 1000]]]\n",
      "\n",
      "\n",
      "[[[  42    7    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5   10   15]\n",
      "  [  10  100 1000]]]\n",
      "//\n",
      "[[[3 3 3]\n",
      "  [1 3 5]]\n",
      "\n",
      " [[1 5 5]\n",
      "  [1 1 1]]]\n",
      "=\n",
      "[[[  14    2    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5    2    3]\n",
      "  [  10  100 1000]]]\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def examine_output(pipe_out):\n",
    "    l = pipe_out[0].as_array()\n",
    "    r = pipe_out[1].as_array()\n",
    "    sum_out = pipe_out[2].as_array()\n",
    "    mul_out = pipe_out[3].as_array()\n",
    "    div_out = pipe_out[4].as_array()\n",
    "    print(\"{}\\n+\\n{}\\n=\\n{}\\n\\n\".format(l, r, sum_out))\n",
    "    print(\"{}\\n*\\n{}\\n=\\n{}\\n\\n\".format(l, r, mul_out))\n",
    "    print(\"{}\\n//\\n{}\\n=\\n{}\\n\\n\".format(l, r, div_out))\n",
    "\n",
    "examine_output(out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see the resulting tensors are obtained by applying the arithmetic operation between corresponding elements of its inputs.\n",
    "\n",
    "The shapes of the arguments to arithmetic operators should match (with an exception for scalar tensor inputs that we will describe in the next section), otherwise we will get an error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constant and scalar operands\n",
    "Until now we considered only tensor inputs of matching shapes for inputs of arithmetic operators. DALI allows one of the operands to be a constant or a tensor consisting of scalars. They can appear on both sides of binary expression.\n",
    "\n",
    "## Constants\n",
    "In `define_graph` step, constant operand for arithmetic operator can be: values of Python's `int` and `float` types used directly, or those values wrapped in `nvidia.dali.types.Constant`. Operation between tensor and constant results in the constant being broadcasted to all elements of the tensor. The same costant is used with all samples in the batch.\n",
    "\n",
    "*Note: Currently all values of integral constants are passed internally to DALI as int32 and all values of floating point constants are passed to DALI as float32.*\n",
    "\n",
    "The Python `int` values will be treated as `int32` and the `float` as `float32` in regard to type promotions.\n",
    "\n",
    "The DALI `Constant` can be used to indicate other types. It accepts `DALIDataType` enum values as second argument and has convenience member functions like `.uint8()` or `.float32()` that can be used for conversions.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the Constants\n",
    "\n",
    "Let's adjust the Pipeline to utilize constants first. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    " class ArithmeticConstantsPipeline(Pipeline):                   \n",
    "    def __init__(self, batch_size, num_threads, device_id):\n",
    "        super(ArithmeticConstantsPipeline, self).__init__(batch_size, num_threads, device_id, seed=12)\n",
    "        self.left_source = ops.ExternalSource()\n",
    "        self.right_source = ops.ExternalSource()\n",
    "\n",
    "    def define_graph(self):                                                                \n",
    "        self.left = self.left_source()\n",
    "        self.right = self.right_source()\n",
    "        add_200 = self.left + 200\n",
    "        mul_075 = self.left * 0.75\n",
    "        sub_15 = Constant(15).float32() - self.right\n",
    "        return self.left, self.right, add_200, mul_075, sub_15\n",
    "\n",
    "    def iter_setup(self):\n",
    "        # Get the data batches with selected types\n",
    "        (l, r) = get_data()\n",
    "        # Feed it to external source\n",
    "        self.feed_input(self.left, l)\n",
    "        self.feed_input(self.right, r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = ArithmeticConstantsPipeline(batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "pipe.build()\n",
    "out = pipe.run()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to display the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[  42    7    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5   10   15]\n",
      "  [  10  100 1000]]]\n",
      "+ 200 =\n",
      "[[[ 242  207  200]\n",
      "  [ 200  200  200]]\n",
      "\n",
      " [[ 205  210  215]\n",
      "  [ 210  300 1200]]]\n",
      "\n",
      "\n",
      "[[[  42    7    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5   10   15]\n",
      "  [  10  100 1000]]]\n",
      "* 0.75 =\n",
      "[[[ 31.5    5.25   0.  ]\n",
      "  [  0.     0.     0.  ]]\n",
      "\n",
      " [[  3.75   7.5   11.25]\n",
      "  [  7.5   75.   750.  ]]]\n",
      "\n",
      "\n",
      "15 -\n",
      "[[[3 3 3]\n",
      "  [1 3 5]]\n",
      "\n",
      " [[1 5 5]\n",
      "  [1 1 1]]]\n",
      "=\n",
      "[[[12. 12. 12.]\n",
      "  [14. 12. 10.]]\n",
      "\n",
      " [[14. 10. 10.]\n",
      "  [14. 14. 14.]]]\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def examine_output(pipe_out):\n",
    "    l = pipe_out[0].as_array()\n",
    "    r = pipe_out[1].as_array()\n",
    "    add_200 = pipe_out[2].as_array()\n",
    "    mul_075 = pipe_out[3].as_array()\n",
    "    sub_15 = pipe_out[4].as_array()\n",
    "    print(\"{}\\n+ 200 =\\n{}\\n\\n\".format(l, add_200))\n",
    "    print(\"{}\\n* 0.75 =\\n{}\\n\\n\".format(l, mul_075))\n",
    "    print(\"15 -\\n{}\\n=\\n{}\\n\\n\".format(r, sub_15))\n",
    "\n",
    "examine_output(out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see the constant operand is broadcasted to all elements of all tensors in the batch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scalars\n",
    "\n",
    "As an addition to element-wise operation between Tensors of the same sizes, DALI allows to use tensors containing scalar values - that is a batch of elements with shape `{1}` as one of the operands. The tensor of scalars will behave similarly to a constant described above - each scalar value will be broadcasted to every element of the other tensor operand. Note that contrary to constants, the tensor is still as a batch of elements - each scalar value is broadcasted to corresponding tensor.\n",
    "\n",
    "### Using scalar tensors\n",
    "\n",
    "We will use `Uniform` Operator to generate a batch of random scalar values, that we will use in our example.\n",
    "We already defined our inputs to be a batch of tensors with shape = `{2 x 3}`, so we will be able to observe how the single constant or scalar values are propagated to all elements of the second operand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ArithmeticScalarsPipeline(Pipeline):                   \n",
    "    def __init__(self, batch_size, num_threads, device_id):\n",
    "        super(ArithmeticScalarsPipeline, self).__init__(batch_size, num_threads, device_id, seed=12)\n",
    "        self.tensor_source = ops.ExternalSource()\n",
    "        self.uniform = ops.Uniform(range=[-10, 10])\n",
    "\n",
    "    def define_graph(self):                                                                \n",
    "        self.tensor = self.tensor_source()\n",
    "        uni = self.uniform()\n",
    "        return self.tensor, uni, self.tensor * uni\n",
    "\n",
    "    def iter_setup(self):\n",
    "        # Get the data batches with selected types, we only use one input\n",
    "        (t, _) = get_data()\n",
    "        # Feed it to external source\n",
    "        self.feed_input(self.tensor, t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to build and run the Pipeline. It will allow to scale our input by some random numbers generated by the `Uniform` Operator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = ArithmeticScalarsPipeline(batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "pipe.build()\n",
    "out = pipe.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[  42    7    0]\n",
      "  [   0    0    0]]\n",
      "\n",
      " [[   5   10   15]\n",
      "  [  10  100 1000]]]\n",
      "*\n",
      "[[3.9766016]\n",
      " [0.666193 ]]\n",
      "=\n",
      "[[[167.01727   27.836212   0.      ]\n",
      "  [  0.         0.         0.      ]]\n",
      "\n",
      " [[  3.330965   6.66193    9.992895]\n",
      "  [  6.66193   66.6193   666.193   ]]]\n"
     ]
    }
   ],
   "source": [
    "def examine_output(pipe_out):\n",
    "    t = pipe_out[0].as_array()\n",
    "    uni = pipe_out[1].as_array()\n",
    "    scaled = pipe_out[2].as_array()\n",
    "    print(\"{}\\n*\\n{}\\n=\\n{}\".format(t, uni, scaled))\n",
    "\n",
    "examine_output(out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how we multiply corresponding elements of two batches (consisting of two elements here). Their shapes do not match, but as the second one operand is a batch two of `{1}`-shaped tensors, it is considered a scalar input."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}