You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sungwoo Park <gl...@gmail.com> on 2019/10/18 09:15:30 UTC

Announce: MR3 0.10 released

I am pleased to announce the release of MR3 0.10. New features are:

* TaskScheduler supports a new scheduling policy (specified by
mr3.taskattempt.queue.scheme) which significantly improves the throughput
for concurrent queries. From our internal experiments using TPC-DS
benchmarks, the new scheduling policy increases the throughput 20 to 30
percent at concurreny level 16 to 32. Now Hive on MR3 (based on Hive 3.1.2)
delivers almost twice the throughput of Hive-LLAP included in HDP 3.

* Compaction sends DAGs to MR3, instead of MapReduce, when
hive.mr3.compaction.using.mr3 is set to true. As it no longer depends on
MapReduce for compaction, Hive on MR3 supports compaction on Kubernetes as
well.

* Helm charts are supported on Kubernetes.

* MR3 supports Hive 3.1.2 and 2.3.6. It also supports Hive 4.0.0-SNAPSHOT.

* Scritps for running HPL/SQL are included, both for Hadoop and Kuberentes.

* LlapDecider and ConvertJoinMapJoion directly asks MR3 DAGAppMaster for
the current number of Task slots (similar to executors in Hive-LLAP) and
nodes.

* DAGAppMaster recovers from OutOfMemoryErrors due to the exhaustion of
threads.

You can download MR3 0.10 at:

https://mr3.postech.ac.kr/download/home/

Cheers,

--- Sungwoo Park