You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by "second_comet@yahoo.com" <se...@yahoo.com> on 2022/06/21 01:19:05 UTC

hive with spark backend engine

Hello team,
The default Hive is using Hadoop map reduce. May i know anyone successfully swap the engine by running spark operator/cluster? Any guide or example on this?
Thank you,Teoh

Re: hive with spark backend engine

Posted by Rob Verkuylen <ro...@verkuylen.net>.

Hive On Spark has never been a one on one replacement for Hive on MR. Hive
on Spark has/had several issues and could not fully cover all use-cases.
Therefore Hive On Spark has pretty much been deprecated in favour of Hive
on Tez. It offers the scalability advantages of Hive on MR with much of the
performance that Spark could bring, but with a much better integration for
query planner optimization.

On Tue, Jun 21, 2022 at 12:52 PM Yong Walt <yo...@gmail.com> wrote:

> we have been running hive3 with tez engine.
>
> On Tue, Jun 21, 2022 at 9:19 AM second_comet@yahoo.com <
> second_comet@yahoo.com> wrote:
>
>> Hello team,
>>
>> The default Hive is using Hadoop map reduce. May i know anyone
>> successfully swap the engine by running spark operator/cluster? Any guide
>> or example on this?
>>
>> Thank you,
>> Teoh
>>
>>
>>
>>

Re: hive with spark backend engine

Posted by Yong Walt <yo...@gmail.com>.

we have been running hive3 with tez engine.

On Tue, Jun 21, 2022 at 9:19 AM second_comet@yahoo.com <
second_comet@yahoo.com> wrote:

> Hello team,
>
> The default Hive is using Hadoop map reduce. May i know anyone
> successfully swap the engine by running spark operator/cluster? Any guide
> or example on this?
>
> Thank you,
> Teoh
>
>
>
>