Overview

The Qualification tool analyzes Spark events generated from CPU based Spark applications to help quantify the expected acceleration of migrating a Spark application to GPU. The Profiling tool analyzes both CPU or GPU generated event logs and generates information which can be used for debugging and profiling Spark applications.

Before running this Lab, make sure that any of the previous three labs have been run and the event log files are generated, since the Qualification/Profiling tools take event logs as input. The event logs files are in the /data/eventlogs directory by default, if you move them to another directory, please make sure to update the directory in /home/spark/lp-run-qua-pro.sh script.

1. Connect to System Console using the left-hand menu link.

2. Run the /home/spark/lp-run-qua-pro.sh script, see usage below:

   Copy

   Copied!

               
   
               
   Usage: lp-run-qua-pro.sh <qualification|profiling> <app-eventlog1,app-eventlog2 ...>
   
           Example commands:
           Use profiling tool to process event logs
   
           # the input files can be either CPU or GPU event logs
           bash lp-run-qua-pro.sh profiling spark-application-1660204384913,spark-application-1659761211360
   
           Use qualification tool to process event logs
   
           # the input files can only be CPU event logs
           bash lp-run-qua-pro.sh qualification spark-application-1660123133461,spark-application-1660204645329
           

   
   

3. Copy and paste is available on the Desktop VNC connection. You will see a sidebar on the left of the screen and once that is opened you can paste into the clipboard. Once you have pasted something it is immediately available to paste within the VNC desktop