You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Sungwoo Park <gl...@gmail.com> on 2023/06/01 04:59:21 UTC

Comparison of Trino, Spark, and Hive-MR3

Hi everyone,

We published an article on the performance and correctness of Trino, Spark,
and Hive-MR3, and thought that it could be of interest to Spark users.

https://www.datamonad.com/post/2023-05-31-trino-spark-hive-performance-1.7/

Omitted in the article is the performance of Spark 2.3.1 vs 2.4.0. On the
same 10TB TPC-DS benchmark:

With Spark 3.2.1, it takes 27104 seconds to complete all 99 queries.
With Spark 3.4.0, it takes 19669 seconds to complete all 99 queries.

In both cases, all the queries return correct results.

Thanks,

--- Sungwoo