VPI - Vision Programming Interface

3.2 Release

PyTorch/CUDA interoperability

Overview

This sample shows how interoperability between VPI and PyTorch works in Python. It shows a simple set of image processing operations applied on a sample input image in Pytorch and VPI without any memory copies during the interoperation.

This sample uses PyTorch as an example, however, this interoperability is possible with any library that supports the cuda array interface.

Libraries such as:

The sample starts with an image given by you, converted to grayscale, then performs the following sequence of operations to it:

  1. Lower the image intensity using PyTorch
  2. Does PyTorch -> VPI interoperability without memory copies
    vpi_image = vpi.asimage(torch_image)
  3. Applies a box filter using VPI
  4. Does VPI -> PyTorch interoperability without memory copies
    with vpi_image.rlock_cuda() as cuda_buffer:
    torch_image = torch.as_tensor(cuda_buffer)
    or, with an additional deep memory copy
    torch_image = torch.as_tensor(vpi_image.cuda())
  5. Increase the image intensity using PyTorch

The result is then saved to disk.

Instructions

The usage is:

python3 main.py <input image>

where

  • input image: input image file name; it accepts png, jpeg and possibly others.

Here's one example:

  • Python
    python3 main.py ../assets/kodim08.png

The sample will produce vpi_pytorch.png as output file.

Results

Input imageOutput image

Source Code

For convenience, here's the code that is also installed in the samples directory.

27 import vpi
28 import torch
29 import numpy as np
30 
31 from PIL import Image, ImageOps
32 from argparse import ArgumentParser
33 
34 # Parse command line arguments
35 parser = ArgumentParser()
36 parser.add_argument('input', help='Image to be used as input')
37 
38 args = parser.parse_args()
39 
40 # Make sure CUDA is available for this example
41 assert torch.cuda.is_available()
42 cuda_device = torch.device('cuda')
43 
44 # Read the input image and convert it to grayscale
45 try:
46  pil_image = ImageOps.grayscale(Image.open(args.input))
47 except IOError:
48  sys.exit("Input file not found")
49 except:
50  sys.exit("Error with input file")
51 
52 # Pillow -> NumPy --------------------------------------
53 np_image = np.asarray(pil_image)
54 
55 # NumPy -> PyTorch/CUDA --------------------------------
56 torch_image = torch.asarray(np_image).cuda()
57 
58 # Perform an operation using PyTorch
59 torch_image = torch_image/255 * 0.5
60 
61 # PyTorch/CUDA -> VPI, no copies involved --------------
62 vpi_image = vpi.asimage(torch_image)
63 
64 # Peform operations using VPI's CUDA backend
65 with vpi.Backend.CUDA:
66  # Blur the input image with box filter
67  vpi_image = vpi_image.box_filter(3)
68 
69 # VPI -> PyTorch/CUDA, no copies involved --------------
70 with vpi_image.rlock_cuda() as cuda_buffer:
71  # Perform another operation using PyTorch
72  torch_tensor = torch.as_tensor(cuda_buffer, device=cuda_device)
73  torch_image = torch_tensor*255 + 128
74 
75 # PyTorch -> NumPy -------------------------------------
76 np_image = np.asarray(torch_image.cpu())
77 
78 # NumPy -> Pillow --------------------------------------
79 pil_image = Image.fromarray(np_image)
80 
81 # Save the output to disk
82 pil_image.convert('L').save('vpi_pytorch.png')