Comet Benchmarking Guide#

To track progress on performance, we regularly run benchmarks derived from TPC-H and TPC-DS.

The benchmarking scripts are contained here.

Data generation scripts are available in the DataFusion Benchmarks GitHub repository.

Current Benchmark Results#

• Benchmarks derived from TPC-H

• Benchmarks derived from TPC-DS

Benchmarking Guides#

Available benchmarking guides:

• Benchmarking on macOS

• Benchmarking on AWS EC2

• TPC-DS Benchmarking with spark-sql-perf

We also have many micro benchmarks that can be run from an IDE located here.