Overview
Welcome to the trial of NVIDIA AI Workflows on NVIDIA LaunchPad.
NVIDIA AI Workflows are available as part of NVIDIA AI Enterprise – an end-to-end, secure, cloud-native suite of AI software, enabling organizations to solve new challenges while increasing operational efficiency. Organizations start their AI journey by using the open, freely available NGC libraries and frameworks to experiment and pilot. When they’re ready to move from pilot to production, enterprises can easily transition to a fully managed and secure AI platform with an NVIDIA AI Enterprise subscription. This gives enterprises deploying business-critical AI the assurance of business continuity with NVIDIA Enterprise Support and access to NVIDIA AI experts.
Within this LaunchPad lab, you will gain experience with AI workflows that can accelerate your path to AI outcomes. These are packaged AI workflow examples that include NVIDIA SDKs, AI frameworks, and pre-trained models, as well as resources such as helm charts, Jupyter notebooks, and documentation to help you get started in building AI-based solutions. NVIDIA’s cloud-native AI workflows run as microservices that can be deployed on Kubernetes alone or with other microservices to create production-ready applications.
Key Benefits:
Reduce development time at a lower cost
Improve accuracy and performance
Gain confidence in outcomes by leveraging NVIDIA expertise
NVIDIA AI Workflows are intended to provide reference solutions for leveraging NVIDIA frameworks to build AI solutions for solving common use cases. These workflows guide fine-tuning and AI model creation to build upon NVIDIA frameworks. The pipelines to create applications are highlighted, as well as guidance on deploying customized applications and integrating them with various components typically found in enterprise environments, such as components for orchestration and management, storage, security, networking, etc.
NVIDIA AI Workflows are available on NVIDIA NGC for NVIDIA AI Enterprise software customers.
NVIDIA AI Workflows are deployed as a package containing the AI framework and tools for automating a cloud-native solution. AI Workflows also have packaged components that include enterprise-ready implementations with best practices that ensure reliability, security, performance, scalability, and interoperability while allowing a path for you to deviate.
A typical workflow may look like the following diagram:
The components and instructions used in the workflow are intended to be used as examples for integration and may need to be sufficiently production-ready on their own, as stated. The workflow should be customized and integrated into one’s infrastructure, using the workflow as a reference. For example, all of the instructions in these workflows assume a single node infrastructure, whereas production deployments should be performed in a high availability (HA) environment.
This reference AI workflow for route optimization shows how to use NVIDIA cuOpt GPU accelerated solver. Routing optimization is prevalent in many industries, where determining the most cost-effective route can contribute significant cost savings for meal delivery, where a single restaurant franchise can deliver millions of meals a day, or a telecommunications company that dispatches millions of jobs per year. In these large-scale scenarios, inefficient routes can cost billions of dollars in operational costs and reduce our environmental carbon footprint.
Route optimization problems such as those described above are commonly known as the Traveling Salesperson (TSP) problem. To reduce the time to develop a GPU-accelerated TSP solution, NVIDIA has developed the route optimization workflow to streamline the development of Vehicle Routing Problem (VRP) solutions.
This NVIDIA AI Workflow contains details on how to deploy a sample opinionated AI solution for route optimization; the following items are included:
Origin-destination cost-matrix creation
Data preprocessing
NVIDIA cuOpt™ GPU accelerated solver pipeline
Driving directions
Map visualization
Components for authentication, logging, and monitoring the workflow
Cloud Native deployable bundle packaged as a helm chart
Using the above assets, this NVIDIA AI Workflow provides a reference for you to get started and build your AI solution with minimal preparation and includes enterprise-ready implementation best practices which range from authentication, monitoring, reporting, and load balancing, helping you achieve the desired AI outcome more quickly while still allowing a path for you to deviate.
NVIDIA AI Workflows are designed as microservices. They can be deployed on Kubernetes alone or with other microservices to create a production-ready application for seamless scaling in your enterprise environment.
The following cloud-native Kubernetes services are used with this workflow:
NVIDIA Merlin
Prometheus
Grafana
These components are used to build and deploy training and inference pipelines, integrated with the additional components as indicated in the below diagram:
The following sections describe these NVIDIA AI components further.
Data Preparation
The NVIDIA route optimization workflow uses the NVIDIA cuOpt server via a representational state transfer (REST) microservice API to generate routes. To do this, a series of sample synthetic datasets are included within the workflow to assign orders to a fleet of delivery drivers.
In this workflow, we are using locations from the following Kaggle Kaggle dataset dataset . This is a dataset of Airbnbs in New York City and will act as stops for our fleet delivery drivers. There are 100 locations total which 97 orders must be delivered to and the orders are originating from 3 depots. We have 15 vehicles available.
Three CSV files are used by workflow to assign the drivers to their appropriate orders; Orders, Depot and Route.
- Order
- Depot
- Route
An Order can be delivered to a customer, a pickup from a customer, or some other type of work. Examples include a furniture delivery, a grease pickup from a restaurant, or an inspection visit. This workflow looks at deliveries from a distribution center to stores. The Order dataset includes the stores’ data. This includes store name, location, start and end time (store hours), demand (order weight in pounds), and service time (how long it will take to unload the package).
A Depot is a location that a vehicle departs from at the beginning of its workday and returns to at the end of the workday. Depots are locations where the vehicles are loaded (for deliveries) or unloaded (for pickups). Sometimes, a depot can also act as a renewal location whereby the vehicle can unload, reload, and continue performing deliveries and pickups. A Depot has open and close times, as specified by a hard time window. Vehicles can’t arrive at a Depot outside this time window. In this route optimization workflow, vehicles depart the depot in the morning and return at the end of the day. The depots’ information includes names, locations, and start and end times (operation hours).
Route information specifies the vehicle and driver characteristics, such as the vehicle capacity, work shift hours, and driving range, and it represents the traversal between depots and orders. The features needed here are vehicle name/ID number, start and end depot name, start and end time (vehicle/driver shift hours), and carrying capacity.
The sample AI Workflow uses a combination of these three CSVs to find the best cost-effective route using your data for your specific use case. For example, within the Order CSV file, the package weight is indicated, and the Route CSV contains the route of the delivery truck with the maximum order weight. The Route is assigned to a Depot.
You might have additional features depending on the problem, such as order priorities or vehicle break time windows. Other features would be preprocessed similarly to the features shown in the workflow.
The sample AI Workflow uses a combination of these three CSVs to find the best cost-effective route using your data for your specific use case. For example, within the Order CSV file, the package weight is indicated, and the Route CSV contains the route of the delivery truck with the maximum order weight. The Route is assigned to a Depot.
Running the workflow
The cuOpt Solver pipeline includes data preprocessing, creating a cost matrix, calling on the cuOpt solver, and mapping the routes. This is all done within a Jupyter Notebook client.
- Data Preprocessing
- Route Mapping
The cuOpt server has a set of data requirements handled in the pre-processing stage that includes modeling the data as arrays of integers and creating a cost matrix. This is done in the Jupyter notebook client, where the route optimization workflow uses the Open Source Routing Machine (OSRM) API as an open-source router that uses OpenStreetMap. We use OSRM to build the cost matrix that represents the travel time from one depot or order to another. Once the preprocessing stage is complete, the data from the three datasets mentioned above and the cost matrix are sent over and imported to the cuOpt Server via API calls, again made using the Jupyter notebook client.
Below is an example cost matrix. If a problem has five total locations, then the cost matrix, in the form of a dataframe, will look like this.
This cost matrix represents the travel time in minutes, as used in the workflow, such that the traveling from location 0 to location 1 takes 16.546667 minutes. Note that the cost of going from a location to itself is typically 0, and the cost of going from location A to location B is not necessarily equal to going from location B to location A.
Once the cuOpt solver returns-optimized routes, the route optimization workflow uses OSRM again to visualize the routes. OSRM parses the cuOpt solver response, converts locations to coordinate points, and then maps the routes. These optimized routes inherently include driving directions for each order.
To assist you in your LaunchPad journey, there are a couple of important links on the left-hand navigation pane of this page. The lab steps in the next few sections will reference these links.