Benchmark Environments#

Cluster information for the ETL benchmarks used for the qualification estimation model. Note that all benchmarks were run using the NDS benchmark at SF3K (3 TB scale).#

Environment

CPU Cluster

GPU Cluster

On-prem

8x 128-core

8x 128-core + 8x A100 40 GB

Dataproc

8x n1-standard-16

8x g2-standard-16

EMR

8x m7gd.8xlarge

8x g6.8xlarge

Databricks AWS

8x m7gd.8xlage

8x g5.8xlarge

Databricks Azure

8x E8ds_v4

8x NC8as_T4_v3