The Microbenchmarks Lab is derived from the spark-rapids-example repo. The microbenchmark on RAPIDS Accelerator For Apache Spark is to identify, test and analyze the best queries which can be accelerated on the GPU.
The queries are based on several tables in Parquet format derived from the TPC-DS benchmark, so that similar speedups can be reproducible by others. The microbenchmarks include commonly used Spark SQL operations such as expand, hash aggregate, windowing, and cross join and runs the same queries in CPU mode and GPU mode. You can see some queries are faster the second time, which can be caused by JVM JIT, initialization overhead or caching input data in the OS page cache, etc. The improved performance is influenced by many components, including the dataset’s scale factors and the GPU accelerator model.
Connect to System Console using the left-hand navigation menu link.
Find current IP address of Spark-RAPIDS pod.
kubectl describe pod sparkrunner-0 | grep IP
Connect to the sparkrunner pod.
kubectl exec --stdin --tty sparkrunner-0 -- /bin/bash
Update the
/home/spark/lp-runjupyter-etl-gpu.shand/home/spark/lp-runjupyter-etl-cpu.shscript and replace the following line with the correct IP.SPARK_DRIVER_HOST=192.168.<IP>.<IP>
Copy and paste is available on the Desktop VNC connection. You will see a sidebar on the left of the screen and once that is opened you can paste into the clipboard. Once you have pasted something it is immediately available to paste within the VNC desktop
