You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sungwoo Park <gl...@gmail.com> on 2024/03/19 13:11:13 UTC
Hive-MR3 1.10 released
Hello Hive users,
We have released Hive on MR3 1.10. MR3 is an execution engine similar to
MapReduce and Tez, and it supports Hadoop, Kubernetes, and standalone mode.
Hive-MR3 uses MR3 for its execution backend in Hive 3.1.3. If you are
interested, please give it a try.
In MR3 1.10, we have re-written the shuffle library in Tez. In the previous
version, all tasks manage fetchers independently of each other. Now all
fetchers inside a container are managed by a common shuffle server.
For those interested in performance comparison, here are the latest results
of testing Hive-MR3 1.9/1.10, Trino 435, and Spark 3.4.1 using the
(original) TPC-DS benchmark with 10TB scale. All the systems were tested
with Java 17.
Hive-MR3 1.9: total 6473 seconds, geo-mean 25.0 seconds.
Hive-MR3 1.10: total 6138 seconds, geo-mean 24.4 seconds.
Trino 435: total 6950 seconds, geo-mean 19.2 seconds. Query 23 returns
wrong results. Query 72 fails.
Spark 3.4.1 (using Parquet instead of ORC): total 19044 seconds, geo-mean
35.9 seconds.
Thank you,
--- Sungwoo