You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/09/29 16:31:45 UTC

(datafusion-comet) branch gh-readonly-queue/main/pr-6308-669fbde38e326b3076821ab9604db4f21b61e930 created (now e3d52feaf6)

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-6308-669fbde38e326b3076821ab9604db4f21b61e930
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git


      at e3d52feaf6 docs: update TPC-DS benchmark results for Comet 1.1.0 on Spark 4.2 (#6308)

This branch includes the following new commits:

     new e3d52feaf6 docs: update TPC-DS benchmark results for Comet 1.1.0 on Spark 4.2 (#6308)

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org


(datafusion-comet) 01/01: docs: update TPC-DS benchmark results for Comet 1.1.0 on Spark 4.2 (#6308)

Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6308-669fbde38e326b3076821ab9604db4f21b61e930
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git

commit e3d52feaf6584295f6870a2b72d95ebf83a18a05
Author: Andy Grove <ag...@apache.org>
AuthorDate: Tue Sep 29 03:05:50 2026 +0000

    docs: update TPC-DS benchmark results for Comet 1.1.0 on Spark 4.2 (#6308)
---
 README.md                                          |  10 +-
 benchmarks/results/1.1.0/comet-tpcds.json          | 333 +++++++++++++++++++++
 benchmarks/results/1.1.0/spark-tpcds.json          | 326 ++++++++++++++++++++
 .../benchmark-results/1.1.0/tpcds_allqueries.png   | Bin 0 -> 23791 bytes
 .../1.1.0/tpcds_queries_compare.png                | Bin 0 -> 60485 bytes
 .../1.1.0/tpcds_queries_speedup_abs.png            | Bin 0 -> 93319 bytes
 .../1.1.0/tpcds_queries_speedup_rel.png            | Bin 0 -> 120728 bytes
 .../contributor-guide/benchmark-results/tpc-ds.md  |  11 +-
 8 files changed, 670 insertions(+), 10 deletions(-)

diff --git a/README.md b/README.md
index 6c57277eee..249125cb2b 100644
--- a/README.md
+++ b/README.md
@@ -42,17 +42,17 @@ pipeline, operators and expressions execute as Rust code (via the [Apache DataFu
 or as JVM code that operates directly on Arrow batches. Comet integrates with the Spark ecosystem
 without requiring any code changes.
 
-**Comet provides a ~2x speedup for TPC-DS @ SF 1000 (1TB), resulting in ~50% cost savings.**
+**Comet provides a ~1.8x speedup over Spark 4.2 for TPC-DS @ SF 1000 (1TB), resulting in ~45% cost savings.**
 
-That 2x speedup gives you a choice: finish the same Spark workload in half the time on the cluster you already have,
-or match your current Spark performance on roughly half the resources. Either way, the gain translates directly into
+That speedup gives you a choice: finish the same Spark workload in just over half the time on the cluster you already
+have, or match your current Spark performance on just over half the resources. Either way, the gain translates directly into
 lower cloud bills, reduced on-prem capacity, and lower energy usage, with no changes to your existing Spark SQL,
 DataFrame, or PySpark code. Comet runs on commodity hardware: no GPUs, FPGAs, or other specialized accelerators are
 required, so the savings come from better utilization of the infrastructure you already run on.
 
-![](docs/source/_static/images/benchmark-results/1.0.0/tpcds_allqueries.png)
+![](docs/source/_static/images/benchmark-results/1.1.0/tpcds_allqueries.png)
 
-![](docs/source/_static/images/benchmark-results/1.0.0/tpcds_queries_speedup_rel.png)
+![](docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_rel.png)
 
 See the [Comet Benchmarking Guide](https://datafusion.apache.org/comet/contributor-guide/benchmarking.html) for more details.
 
diff --git a/benchmarks/results/1.1.0/comet-tpcds.json b/benchmarks/results/1.1.0/comet-tpcds.json
new file mode 100644
index 0000000000..d8b3025059
--- /dev/null
+++ b/benchmarks/results/1.1.0/comet-tpcds.json
@@ -0,0 +1,333 @@
+{
+  "engine": "comet",
+  "benchmark": "tpcds",
+  "format": "parquet",
+  "arch": "amd64",
+  "tags": {
+    "name": "tpcds",
+    "codec": "zstd",
+    "engine": "comet",
+    "format": "parquet",
+    "shuffle": "auto",
+    "comet_exec": "all",
+    "scale_factor": "1000",
+    "spark_profile": "4.2_2.13"
+  },
+  "spark_conf": {
+    "spark.comet.exec.memoryPool.fraction": "0.8",
+    "spark.comet.explain.fallback.enabled": "true",
+    "spark.comet.explain.fallback.log.enabled": "true",
+    "spark.comet.explain.format": "verbose",
+    "spark.comet.explainFallback.enabled": "true",
+    "spark.comet.logFallbackReasons.enabled": "true"
+  },
+  "1": [
+    7.133
+  ],
+  "2": [
+    6.164999999999999
+  ],
+  "3": [
+    1.8645
+  ],
+  "4": [
+    16.186
+  ],
+  "5": [
+    5.2989999999999995
+  ],
+  "6": [
+    1.472
+  ],
+  "7": [
+    2.5324999999999998
+  ],
+  "8": [
+    1.8295
+  ],
+  "9": [
+    10.258
+  ],
+  "10": [
+    3.5725000000000002
+  ],
+  "11": [
+    8.413
+  ],
+  "12": [
+    0.923
+  ],
+  "13": [
+    2.224
+  ],
+  "15": [
+    1.216
+  ],
+  "16": [
+    5.4235
+  ],
+  "17": [
+    2.6105
+  ],
+  "18": [
+    2.99
+  ],
+  "19": [
+    1.2574999999999998
+  ],
+  "20": [
+    0.879
+  ],
+  "21": [
+    0.6755
+  ],
+  "22": [
+    2.7085
+  ],
+  "25": [
+    1.8675000000000002
+  ],
+  "26": [
+    1.3195000000000001
+  ],
+  "27": [
+    2.1189999999999998
+  ],
+  "28": [
+    12.74
+  ],
+  "29": [
+    4.8665
+  ],
+  "30": [
+    2.141
+  ],
+  "31": [
+    3.738
+  ],
+  "32": [
+    0.872
+  ],
+  "33": [
+    1.173
+  ],
+  "34": [
+    1.569
+  ],
+  "35": [
+    3.2755
+  ],
+  "36": [
+    2.0335
+  ],
+  "37": [
+    2.3914999999999997
+  ],
+  "38": [
+    3.9234999999999998
+  ],
+  "40": [
+    2.3465
+  ],
+  "41": [
+    0.383
+  ],
+  "42": [
+    0.6805
+  ],
+  "43": [
+    1.814
+  ],
+  "44": [
+    7.328
+  ],
+  "45": [
+    1.5405
+  ],
+  "46": [
+    1.7715
+  ],
+  "47": [
+    7.973
+  ],
+  "48": [
+    1.964
+  ],
+  "49": [
+    2.567
+  ],
+  "50": [
+    12.91
+  ],
+  "51": [
+    4.148
+  ],
+  "52": [
+    0.7170000000000001
+  ],
+  "53": [
+    1.8965
+  ],
+  "54": [
+    5.907
+  ],
+  "55": [
+    0.769
+  ],
+  "56": [
+    1.0735
+  ],
+  "57": [
+    2.849
+  ],
+  "58": [
+    1.521
+  ],
+  "59": [
+    5.596500000000001
+  ],
+  "60": [
+    1.1195
+  ],
+  "61": [
+    1.4075
+  ],
+  "62": [
+    2.8925
+  ],
+  "63": [
+    1.459
+  ],
+  "64": [
+    9.9265
+  ],
+  "65": [
+    4.1105
+  ],
+  "66": [
+    2.7125
+  ],
+  "67": [
+    21.241
+  ],
+  "68": [
+    3.942
+  ],
+  "69": [
+    1.6075
+  ],
+  "70": [
+    3.098
+  ],
+  "71": [
+    1.213
+  ],
+  "72": [
+    11.003
+  ],
+  "73": [
+    0.9795
+  ],
+  "74": [
+    5.795
+  ],
+  "75": [
+    10.972000000000001
+  ],
+  "76": [
+    4.984999999999999
+  ],
+  "77": [
+    1.248
+  ],
+  "78": [
+    12.799
+  ],
+  "79": [
+    1.492
+  ],
+  "80": [
+    4.846500000000001
+  ],
+  "81": [
+    2.4370000000000003
+  ],
+  "82": [
+    3.652
+  ],
+  "83": [
+    0.916
+  ],
+  "84": [
+    2.8200000000000003
+  ],
+  "85": [
+    2.7350000000000003
+  ],
+  "86": [
+    1.1855
+  ],
+  "87": [
+    4.0024999999999995
+  ],
+  "88": [
+    13.398
+  ],
+  "89": [
+    1.573
+  ],
+  "90": [
+    2.516
+  ],
+  "91": [
+    0.6715
+  ],
+  "92": [
+    0.7324999999999999
+  ],
+  "93": [
+    16.158
+  ],
+  "94": [
+    3.8920000000000003
+  ],
+  "95": [
+    8.6085
+  ],
+  "96": [
+    3.0789999999999997
+  ],
+  "97": [
+    3.827
+  ],
+  "98": [
+    1.326
+  ],
+  "99": [
+    2.918
+  ],
+  "14a": [
+    20.4705
+  ],
+  "14b": [
+    21.137999999999998
+  ],
+  "23a": [
+    26.637500000000003
+  ],
+  "23b": [
+    33.363
+  ],
+  "24a": [
+    19.6955
+  ],
+  "24b": [
+    18.195999999999998
+  ],
+  "39a": [
+    2.4355
+  ],
+  "39b": [
+    2.09
+  ]
+}
\ No newline at end of file
diff --git a/benchmarks/results/1.1.0/spark-tpcds.json b/benchmarks/results/1.1.0/spark-tpcds.json
new file mode 100644
index 0000000000..538b72f2df
--- /dev/null
+++ b/benchmarks/results/1.1.0/spark-tpcds.json
@@ -0,0 +1,326 @@
+{
+  "engine": "spark",
+  "benchmark": "tpcds",
+  "format": "parquet",
+  "arch": "amd64",
+  "tags": {
+    "name": "tpcds",
+    "codec": "zstd",
+    "engine": "spark",
+    "format": "parquet",
+    "shuffle": "auto",
+    "comet_exec": "all",
+    "scale_factor": "1000",
+    "spark_profile": "4.2_2.13"
+  },
+  "spark_conf": {},
+  "1": [
+    7.376
+  ],
+  "2": [
+    7.022
+  ],
+  "3": [
+    1.7770000000000001
+  ],
+  "4": [
+    38.605999999999995
+  ],
+  "5": [
+    11.1095
+  ],
+  "6": [
+    1.818
+  ],
+  "7": [
+    5.0025
+  ],
+  "8": [
+    2.4625
+  ],
+  "9": [
+    13.8935
+  ],
+  "10": [
+    3.6885000000000003
+  ],
+  "11": [
+    14.589500000000001
+  ],
+  "12": [
+    1.671
+  ],
+  "13": [
+    4.633
+  ],
+  "15": [
+    2.183
+  ],
+  "16": [
+    9.222
+  ],
+  "17": [
+    4.5545
+  ],
+  "18": [
+    3.412
+  ],
+  "19": [
+    1.6545
+  ],
+  "20": [
+    1.6404999999999998
+  ],
+  "21": [
+    1.3885
+  ],
+  "22": [
+    8.725999999999999
+  ],
+  "25": [
+    3.8495
+  ],
+  "26": [
+    2.441
+  ],
+  "27": [
+    2.7615
+  ],
+  "28": [
+    17.589
+  ],
+  "29": [
+    8.411999999999999
+  ],
+  "30": [
+    3.42
+  ],
+  "31": [
+    5.6675
+  ],
+  "32": [
+    1.3010000000000002
+  ],
+  "33": [
+    2.607
+  ],
+  "34": [
+    2.173
+  ],
+  "35": [
+    5.8255
+  ],
+  "36": [
+    2.63
+  ],
+  "37": [
+    3.6875
+  ],
+  "38": [
+    11.376999999999999
+  ],
+  "40": [
+    2.8225
+  ],
+  "41": [
+    0.7055
+  ],
+  "42": [
+    0.9339999999999999
+  ],
+  "43": [
+    2.7345
+  ],
+  "44": [
+    10.4825
+  ],
+  "45": [
+    1.984
+  ],
+  "46": [
+    2.8225000000000002
+  ],
+  "47": [
+    7.5280000000000005
+  ],
+  "48": [
+    4.4015
+  ],
+  "49": [
+    4.105
+  ],
+  "50": [
+    22.815
+  ],
+  "51": [
+    8.022
+  ],
+  "52": [
+    0.913
+  ],
+  "53": [
+    2.431
+  ],
+  "54": [
+    2.5415
+  ],
+  "55": [
+    0.916
+  ],
+  "56": [
+    2.9044999999999996
+  ],
+  "57": [
+    6.263
+  ],
+  "58": [
+    1.8795000000000002
+  ],
+  "59": [
+    7.7195
+  ],
+  "60": [
+    2.951
+  ],
+  "61": [
+    4.407500000000001
+  ],
+  "62": [
+    3.292
+  ],
+  "63": [
+    2.393
+  ],
+  "64": [
+    29.7055
+  ],
+  "65": [
+    10.933
+  ],
+  "66": [
+    3.395
+  ],
+  "67": [
+    47.6055
+  ],
+  "68": [
+    2.3785
+  ],
+  "69": [
+    3.6025
+  ],
+  "70": [
+    4.452
+  ],
+  "71": [
+    1.745
+  ],
+  "72": [
+    13.775
+  ],
+  "73": [
+    1.5845
+  ],
+  "74": [
+    15.453500000000002
+  ],
+  "75": [
+    14.626999999999999
+  ],
+  "76": [
+    6.309
+  ],
+  "77": [
+    1.4385
+  ],
+  "78": [
+    30.075000000000003
+  ],
+  "79": [
+    2.75
+  ],
+  "80": [
+    7.768
+  ],
+  "81": [
+    4.8465
+  ],
+  "82": [
+    10.065999999999999
+  ],
+  "83": [
+    1.362
+  ],
+  "84": [
+    3.0595
+  ],
+  "85": [
+    4.3004999999999995
+  ],
+  "86": [
+    3.9114999999999998
+  ],
+  "87": [
+    11.5115
+  ],
+  "88": [
+    16.642000000000003
+  ],
+  "89": [
+    2.7199999999999998
+  ],
+  "90": [
+    3.4604999999999997
+  ],
+  "91": [
+    1.213
+  ],
+  "92": [
+    1.2095
+  ],
+  "93": [
+    39.47
+  ],
+  "94": [
+    6.4239999999999995
+  ],
+  "95": [
+    20.064
+  ],
+  "96": [
+    4.2045
+  ],
+  "97": [
+    10.29
+  ],
+  "98": [
+    2.0
+  ],
+  "99": [
+    3.8625
+  ],
+  "14a": [
+    40.9215
+  ],
+  "14b": [
+    32.479
+  ],
+  "23a": [
+    67.9605
+  ],
+  "23b": [
+    82.065
+  ],
+  "24a": [
+    27.7015
+  ],
+  "24b": [
+    27.421
+  ],
+  "39a": [
+    5.781499999999999
+  ],
+  "39b": [
+    5.702500000000001
+  ]
+}
\ No newline at end of file
diff --git a/docs/source/_static/images/benchmark-results/1.1.0/tpcds_allqueries.png b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_allqueries.png
new file mode 100644
index 0000000000..1f0f1dd03f
Binary files /dev/null and b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_allqueries.png differ
diff --git a/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_compare.png b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_compare.png
new file mode 100644
index 0000000000..e39d8bd96b
Binary files /dev/null and b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_compare.png differ
diff --git a/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_abs.png b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_abs.png
new file mode 100644
index 0000000000..c51c11ea60
Binary files /dev/null and b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_abs.png differ
diff --git a/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_rel.png b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_rel.png
new file mode 100644
index 0000000000..54a16c8d36
Binary files /dev/null and b/docs/source/_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_rel.png differ
diff --git a/docs/source/contributor-guide/benchmark-results/tpc-ds.md b/docs/source/contributor-guide/benchmark-results/tpc-ds.md
index c20d76abd1..53a0e067fc 100644
--- a/docs/source/contributor-guide/benchmark-results/tpc-ds.md
+++ b/docs/source/contributor-guide/benchmark-results/tpc-ds.md
@@ -19,25 +19,26 @@ under the License.
 
 # TPC-DS
 
-The following benchmarks were performed on an EKS cluster (`r6i.24xlarge` instances with EBS storage) with data stored in S3.
+The following benchmarks were performed on an EKS cluster (`r6i.24xlarge` instances with EBS storage) with data stored in S3,
+using Apache Spark 4.2.0 (Scala 2.13) and Comet 1.1.0. Each query was run twice and the mean time is reported.
 
 ## Benchmark Results
 
 Total time to run all queries (lower is better).
 
-![](../../_static/images/benchmark-results/1.0.0/tpcds_allqueries.png)
+![](../../_static/images/benchmark-results/1.1.0/tpcds_allqueries.png)
 
 Per-query breakdown showing the relative performance of Spark and Comet.
 
-![](../../_static/images/benchmark-results/1.0.0/tpcds_queries_compare.png)
+![](../../_static/images/benchmark-results/1.1.0/tpcds_queries_compare.png)
 
 How much Comet accelerates each query in relative terms.
 
-![](../../_static/images/benchmark-results/1.0.0/tpcds_queries_speedup_rel.png)
+![](../../_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_rel.png)
 
 How much Comet accelerates each query in absolute terms.
 
-![](../../_static/images/benchmark-results/1.0.0/tpcds_queries_speedup_abs.png)
+![](../../_static/images/benchmark-results/1.1.0/tpcds_queries_speedup_abs.png)
 
 ## Configuration
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org