{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DALI binary arithmetic operators - type promotions\n",
    "\n",
    "In this example, we will describe the rules regarding type promotions for binary arithmetic operators in DALI. Details on using arithmetic operators in DALI can be found in \"DALI expressions and arithmetic operators\" notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare the test pipeline\n",
    "\n",
    "First, we will prepare the helper code, so we can easily manipulate the types and values that will appear as tensors in the DALI pipeline.\n",
    "\n",
    "We use `from __future__ import division` to allow `/` and `//` as true division and floor division operators.\n",
    "We will be using numpy as source for the custom provided data and we also need to import several things from DALI, needed to create Pipeline and use ExternalSource Operator. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import division\n",
    "import numpy as np\n",
    "from nvidia.dali.pipeline import Pipeline\n",
    "import nvidia.dali.ops as ops            \n",
    "import nvidia.dali.types as types\n",
    "from nvidia.dali.types import Constant\n",
    "\n",
    "batch_size = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining the data\n",
    "\n",
    "As we are dealing with binary operators, we need two inputs. \n",
    "We will create a simple helper function that returns two numpy arrays of given numpy types with arbitrary selected values. It is to make the manipulation of types easy. In an actual scenario the data processed by DALI arithmetic operators would be tensors produced by other Operator containing some images, video sequences or other data.\n",
    "\n",
    "Keep in mind that shapes of both inputs need to match as those will be element-wise operations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "left_magic_values = [42, 8]\n",
    "right_magic_values = [9, 2]\n",
    "\n",
    "def get_data(left_type, right_type):\n",
    "    return ([left_type(left_magic_values)], [right_type(right_magic_values)])\n",
    "\n",
    "batch_size = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining the pipeline\n",
    "\n",
    "The next step is to define the Pipeline. We override `Pipeline.iter_setup`, a method called by the pipeline before every `Pipeline.run`. It is meant to feed the data into `ExternalSource()` operators indicated by `self.left` and `self.right`.\n",
    "The data will be obtained from `get_data` function to which we pass the left and right types. \n",
    "\n",
    "Note, that we do not need to instantiate any additional operators, we can use regular Python arithmetic expressions on the results of other operators in the `define_graph` step.\n",
    "\n",
    "For convenience, we'll wrap the usage of arithmetic operations in a lambda called `operation`, specified when creating the pipeline.\n",
    "\n",
    "`define_graph` will return both our data inputs and the result of applying `operation` to them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    " class ArithmeticPipeline(Pipeline):                   \n",
    "    def __init__(self, operation, left_type, right_type, batch_size, num_threads, device_id):\n",
    "        super(ArithmeticPipeline, self).__init__(batch_size, num_threads, device_id, seed=12)\n",
    "        self.left_source = ops.ExternalSource()\n",
    "        self.right_source = ops.ExternalSource()\n",
    "        self.operation = operation\n",
    "        self.left_type = left_type\n",
    "        self.right_type = right_type\n",
    "\n",
    "    def define_graph(self):                                                                \n",
    "        self.left = self.left_source()\n",
    "        self.right = self.right_source()\n",
    "        return self.left, self.right, self.operation(self.left, self.right)\n",
    "\n",
    "    def iter_setup(self):\n",
    "        (l, r) = get_data(self.left_type, self.right_type)\n",
    "        self.feed_input(self.left, l)\n",
    "        self.feed_input(self.right, r)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Type promotion rules\n",
    "\n",
    "Type promotions for binary operators are described below. The type promotion rules are commutative. They apply to `+`, `-`, `*`, and `//`. The `/` always returns a float32 for integer inputs, and applies the rules below when at least one of the inputs is a floating point number.\n",
    "\n",
    "| Operand Type | Operand Type | Result Type | Additional Conditions |\n",
    "|:------------:|:------------:|:-----------:| --------------------- |\n",
    "| T      | T      | T                |                        |\n",
    "| floatX | T      | floatX           | where T is not a float |\n",
    "| floatX | floatY | float(max(X, Y)) |                        |\n",
    "| intX   | intY   | int(max(X, Y))   |                        |\n",
    "| uintX  | uintY  | uint(max(X, Y))  |                        |\n",
    "| intX   | uintY  | int2Y            | if X <= Y              |\n",
    "| intX   | uintY  | intX             | if X > Y               |\n",
    "\n",
    "`bool` type is considered the smallest unsigned integer type and is treated as `uint1` with respect to the table above.\n",
    "\n",
    "The bitwise binary  `|`, `&`, and `^` operations abide by the same type promotion rules as arithmetic binary operations, but their inputs are restricted to integral types (bool included).\n",
    "\n",
    "Only multiplication `*` and bitwise operations `|`, `&`, `^` can accept two `bool` inputs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the Pipeline\n",
    "\n",
    "Let's create a Pipeline that adds two tensors of type `uint8`, run it and see the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[42  8]] + [[9 2]] = [[51 10]]; \n",
      "\twith types uint8 + uint8 -> uint8\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def build_and_run(pipe, op_name):\n",
    "    pipe.build()                                                        \n",
    "    pipe_out = pipe.run()\n",
    "    l = pipe_out[0].as_array()\n",
    "    r = pipe_out[1].as_array()\n",
    "    out = pipe_out[2].as_array()\n",
    "    print(\"{} {} {} = {}; \\n\\twith types {} {} {} -> {}\\n\".format(l, op_name, r, out, l.dtype, op_name, r.dtype, out.dtype))\n",
    "    \n",
    "pipe = ArithmeticPipeline((lambda x, y: x + y), np.uint8, np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run(pipe, \"+\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how all of the operators behave with different type combinations by generalizing the example above.\n",
    "You can use the `np_types` or `np_int_types` in the loops to see all possible type combinations. To reduce the output we limit ourselves to only few of them. We also set some additional printing options for numpy to make the output more aligned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.set_printoptions(precision=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[42  8]] + [[9 2]] = [[51 10]]; \n",
      "\twith types uint8 + uint8 -> uint8\n",
      "\n",
      "[[42  8]] + [[9 2]] = [[51 10]]; \n",
      "\twith types uint8 + int32 -> int32\n",
      "\n",
      "[[42  8]] + [[9. 2.]] = [[51. 10.]]; \n",
      "\twith types uint8 + float32 -> float32\n",
      "\n",
      "[[42  8]] - [[9 2]] = [[33  6]]; \n",
      "\twith types uint8 - uint8 -> uint8\n",
      "\n",
      "[[42  8]] - [[9 2]] = [[33  6]]; \n",
      "\twith types uint8 - int32 -> int32\n",
      "\n",
      "[[42  8]] - [[9. 2.]] = [[33.  6.]]; \n",
      "\twith types uint8 - float32 -> float32\n",
      "\n",
      "[[42  8]] * [[9 2]] = [[122  16]]; \n",
      "\twith types uint8 * uint8 -> uint8\n",
      "\n",
      "[[42  8]] * [[9 2]] = [[378  16]]; \n",
      "\twith types uint8 * int32 -> int32\n",
      "\n",
      "[[42  8]] * [[9. 2.]] = [[378.  16.]]; \n",
      "\twith types uint8 * float32 -> float32\n",
      "\n",
      "[[42  8]] / [[9 2]] = [[4.67 4.  ]]; \n",
      "\twith types uint8 / uint8 -> float32\n",
      "\n",
      "[[42  8]] / [[9 2]] = [[4.67 4.  ]]; \n",
      "\twith types uint8 / int32 -> float32\n",
      "\n",
      "[[42  8]] / [[9. 2.]] = [[4.67 4.  ]]; \n",
      "\twith types uint8 / float32 -> float32\n",
      "\n",
      "[[42  8]] // [[9 2]] = [[4 4]]; \n",
      "\twith types uint8 // uint8 -> uint8\n",
      "\n",
      "[[42  8]] // [[9 2]] = [[4 4]]; \n",
      "\twith types uint8 // int32 -> int32\n",
      "\n",
      "[[42  8]] // [[9. 2.]] = [[4.67 4.  ]]; \n",
      "\twith types uint8 // float32 -> float32\n",
      "\n",
      "[[42  8]] | [[9 2]] = [[43 10]]; \n",
      "\twith types uint8 | uint8 -> uint8\n",
      "\n",
      "[[42  8]] | [[9 2]] = [[43 10]]; \n",
      "\twith types uint8 | int32 -> int32\n",
      "\n",
      "[[42  8]] & [[9 2]] = [[8 0]]; \n",
      "\twith types uint8 & uint8 -> uint8\n",
      "\n",
      "[[42  8]] & [[9 2]] = [[8 0]]; \n",
      "\twith types uint8 & int32 -> int32\n",
      "\n",
      "[[42  8]] ^ [[9 2]] = [[35 10]]; \n",
      "\twith types uint8 ^ uint8 -> uint8\n",
      "\n",
      "[[42  8]] ^ [[9 2]] = [[35 10]]; \n",
      "\twith types uint8 ^ int32 -> int32\n",
      "\n"
     ]
    }
   ],
   "source": [
    "arithmetic_operations = [((lambda x, y: x + y) , \"+\"), ((lambda x, y: x - y) , \"-\"),\n",
    "                         ((lambda x, y: x * y) , \"*\"), ((lambda x, y: x / y) , \"/\"),\n",
    "                         ((lambda x, y: x // y) , \"//\")]\n",
    "\n",
    "bitwise_operations = [((lambda x, y: x | y) , \"|\"), ((lambda x, y: x & y) , \"&\"),\n",
    "                      ((lambda x, y: x ^ y) , \"^\")]\n",
    "\n",
    "np_types = [np.int8, np.int16, np.int32, np.int64, \n",
    "            np.uint8, np.uint16, np.uint32, np.uint64,\n",
    "            np.float32, np.float64]\n",
    "\n",
    "for (op, op_name) in arithmetic_operations:\n",
    "    for left_type in [np.uint8]:\n",
    "        for right_type in [np.uint8, np.int32, np.float32]:\n",
    "            pipe = ArithmeticPipeline(op, left_type, right_type, batch_size=batch_size, num_threads=2, device_id = 0)\n",
    "            build_and_run(pipe, op_name)\n",
    "        \n",
    "for (op, op_name) in bitwise_operations:\n",
    "    for left_type in [np.uint8]:\n",
    "        for right_type in [np.uint8, np.int32]:\n",
    "            pipe = ArithmeticPipeline(op, left_type, right_type, batch_size=batch_size, num_threads=2, device_id = 0)\n",
    "            build_and_run(pipe, op_name)\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Constants\n",
    "\n",
    "Instead of operating only on Tensor data, DALI expressions can also work with constants. Those can be either values of Python `int` and `float` types used directly, or those values wrapped in `nvidia.dali.types.Constant`. Operation between tensor and constant results in the constant being broadcasted to all elements of the tensor. The same costant is used with all samples in the batch.\n",
    "\n",
    "*Note: Currently all values of integral constants are passed to DALI as int32 and all values of floating point constants are passed to DALI as float32.*\n",
    "\n",
    "The Python `int` values will be treated as `int32` and the `float` as `float32` in regard to type promotions.\n",
    "\n",
    "The DALI `Constant` can be used to indicate other types. It accepts `DALIDataType` enum values as second argument and has convenience member functions like `.uint8()` or `.float32()` that can be used for conversions.\n",
    "\n",
    "As our expressions will consist of a tensor and a constant, we will adjust our previous pipeline and the helper functions - they only need to generate one tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ArithmeticConstantsPipeline(Pipeline):                   \n",
    "    def __init__(self, operation, tensor_data_type,batch_size, num_threads, device_id):\n",
    "        super(ArithmeticConstantsPipeline, self).__init__(batch_size, num_threads, device_id, seed=12)\n",
    "        self.tensor_source = ops.ExternalSource()\n",
    "        self.operation = operation\n",
    "        self.tensor_data_type = tensor_data_type\n",
    "\n",
    "    def define_graph(self):                                                                \n",
    "        self.tensor = self.tensor_source()\n",
    "        return self.tensor, self.operation(self.tensor)\n",
    "\n",
    "    def iter_setup(self):\n",
    "        (t, _) = get_data(self.tensor_data_type, self.tensor_data_type)\n",
    "        self.feed_input(self.tensor, t)\n",
    "        \n",
    "def build_and_run_with_const(pipe, op_name, constant, is_const_left = False):\n",
    "    pipe.build()                                                        \n",
    "    pipe_out = pipe.run()\n",
    "    t_in = pipe_out[0].as_array()\n",
    "    t_out = pipe_out[1].as_array()\n",
    "    if is_const_left:\n",
    "        print(\"{} {} {} = \\n{}; \\n\\twith types {} {} {} -> {}\\n\".format(constant, op_name, t_in, t_out, type(constant), op_name, t_in.dtype, t_out.dtype))\n",
    "    else:\n",
    "        print(\"{} {} {} = \\n{}; \\n\\twith types {} {} {} -> {}\\n\".format(t_in, op_name, constant, t_out, t_in.dtype, op_name, type(constant), t_out.dtype))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, the `ArithmeticConstantsPipeline` can be parametrized with a function taking the only tensor and returning the result of arithmetic operation between that tensor and a constant.\n",
    "\n",
    "We also adjusted our print message.\n",
    "\n",
    "Now we will check all the examples we mentioned at the beginning: `int`, `float` constants and `nvidia.dali.types.Constant`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[42  8]] + 10 = \n",
      "[[52 18]]; \n",
      "\twith types uint8 + <class 'int'> -> int32\n",
      "\n",
      "[[42.  8.]] + 10 = \n",
      "[[52. 18.]]; \n",
      "\twith types float32 + <class 'int'> -> float32\n",
      "\n",
      "[[42  8]] + 42.3 = \n",
      "[[84.3 50.3]]; \n",
      "\twith types uint8 + <class 'float'> -> float32\n",
      "\n",
      "[[42.  8.]] + 42.3 = \n",
      "[[84.3 50.3]]; \n",
      "\twith types float32 + <class 'float'> -> float32\n",
      "\n"
     ]
    }
   ],
   "source": [
    "constant = 10\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: x + constant), np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"+\", constant)\n",
    "\n",
    "constant = 10\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: x + constant), np.float32, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"+\", constant)\n",
    "\n",
    "\n",
    "constant = 42.3\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: x + constant), np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"+\", constant)\n",
    "\n",
    "constant = 42.3\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: x + constant), np.float32, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"+\", constant)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see the value of the constant is applied to all the elements of the tensor to which it is added.\n",
    "\n",
    "Now let's check how to use the DALI Constant wrapper.\n",
    "\n",
    "Passing an `int` or `float` to DALI Constant marks it as `int32` or `float32` respectively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[42  8]] * 10:DALIDataType.INT32 = \n",
      "[[420  80]]; \n",
      "\twith types uint8 * <class 'nvidia.dali.types.Constant'> -> int32\n",
      "\n",
      "10.0:DALIDataType.FLOAT * [[42  8]] = \n",
      "[[420.  80.]]; \n",
      "\twith types <class 'nvidia.dali.types.Constant'> * uint8 -> float32\n",
      "\n"
     ]
    }
   ],
   "source": [
    "constant = Constant(10)\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: x * constant), np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"*\", constant)\n",
    "\n",
    "\n",
    "constant = Constant(10.0)\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: constant * x), np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"*\", constant, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can either explicitly specify the type as a second argument, or use convenience conversion member functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[42  8]] * 10:DALIDataType.UINT8 = \n",
      "[[164  80]]; \n",
      "\twith types uint8 * <class 'nvidia.dali.types.Constant'> -> uint8\n",
      "\n",
      "10:DALIDataType.UINT8 * [[42  8]] = \n",
      "[[164  80]]; \n",
      "\twith types <class 'nvidia.dali.types.Constant'> * uint8 -> uint8\n",
      "\n",
      "10:DALIDataType.UINT8 * [[42  8]] = \n",
      "[[164  80]]; \n",
      "\twith types <class 'nvidia.dali.types.Constant'> * uint8 -> uint8\n",
      "\n"
     ]
    }
   ],
   "source": [
    "constant = Constant(10, types.DALIDataType.UINT8)\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: x * constant), np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"*\", constant)\n",
    "\n",
    "\n",
    "constant = Constant(10.0, types.DALIDataType.UINT8)\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: constant * x), np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"*\", constant, True)\n",
    "\n",
    "\n",
    "constant = Constant(10).uint8()\n",
    "pipe = ArithmeticConstantsPipeline((lambda x: constant * x), np.uint8, batch_size = batch_size, num_threads = 2, device_id = 0)\n",
    "build_and_run_with_const(pipe, \"*\", constant, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Treating tensors as scalars\n",
    "\n",
    "If one of the tensors is considered a scalar input, the same rules apply.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}