You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by kiran lonikar <lo...@gmail.com> on 2015/11/16 07:37:17 UTC
Hive on Spark Vs Spark SQL
I would like to know if Hive on Spark uses or shares the execution code
with Spark SQL or DataFrames?
More specifically, does Hive on Spark benefit from the changes made to
Spark SQL, project Tungsten? Or is it completely different execution path
where it creates its own plan and executes on RDD?
-Kiran
Re: Hive on Spark Vs Spark SQL
Posted by Reynold Xin <rx...@databricks.com>.
No it does not -- although it'd benefit from some of the work to make
shuffle more robust.
On Sun, Nov 15, 2015 at 10:45 PM, kiran lonikar <lo...@gmail.com> wrote:
> So does not benefit from Project Tungsten right?
>
>
> On Mon, Nov 16, 2015 at 12:07 PM, Reynold Xin <rx...@databricks.com> wrote:
>
>> It's a completely different path.
>>
>>
>> On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar <lo...@gmail.com>
>> wrote:
>>
>>> I would like to know if Hive on Spark uses or shares the execution code
>>> with Spark SQL or DataFrames?
>>>
>>> More specifically, does Hive on Spark benefit from the changes made to
>>> Spark SQL, project Tungsten? Or is it completely different execution path
>>> where it creates its own plan and executes on RDD?
>>>
>>> -Kiran
>>>
>>>
>>
>
Re: Hive on Spark Vs Spark SQL
Posted by Reynold Xin <rx...@databricks.com>.
No it does not -- although it'd benefit from some of the work to make
shuffle more robust.
On Sun, Nov 15, 2015 at 10:45 PM, kiran lonikar <lo...@gmail.com> wrote:
> So does not benefit from Project Tungsten right?
>
>
> On Mon, Nov 16, 2015 at 12:07 PM, Reynold Xin <rx...@databricks.com> wrote:
>
>> It's a completely different path.
>>
>>
>> On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar <lo...@gmail.com>
>> wrote:
>>
>>> I would like to know if Hive on Spark uses or shares the execution code
>>> with Spark SQL or DataFrames?
>>>
>>> More specifically, does Hive on Spark benefit from the changes made to
>>> Spark SQL, project Tungsten? Or is it completely different execution path
>>> where it creates its own plan and executes on RDD?
>>>
>>> -Kiran
>>>
>>>
>>
>
Re: Hive on Spark Vs Spark SQL
Posted by kiran lonikar <lo...@gmail.com>.
So does not benefit from Project Tungsten right?
On Mon, Nov 16, 2015 at 12:07 PM, Reynold Xin <rx...@databricks.com> wrote:
> It's a completely different path.
>
>
> On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar <lo...@gmail.com> wrote:
>
>> I would like to know if Hive on Spark uses or shares the execution code
>> with Spark SQL or DataFrames?
>>
>> More specifically, does Hive on Spark benefit from the changes made to
>> Spark SQL, project Tungsten? Or is it completely different execution path
>> where it creates its own plan and executes on RDD?
>>
>> -Kiran
>>
>>
>
Re: Hive on Spark Vs Spark SQL
Posted by kiran lonikar <lo...@gmail.com>.
So does not benefit from Project Tungsten right?
On Mon, Nov 16, 2015 at 12:07 PM, Reynold Xin <rx...@databricks.com> wrote:
> It's a completely different path.
>
>
> On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar <lo...@gmail.com> wrote:
>
>> I would like to know if Hive on Spark uses or shares the execution code
>> with Spark SQL or DataFrames?
>>
>> More specifically, does Hive on Spark benefit from the changes made to
>> Spark SQL, project Tungsten? Or is it completely different execution path
>> where it creates its own plan and executes on RDD?
>>
>> -Kiran
>>
>>
>
Re: Hive on Spark Vs Spark SQL
Posted by Reynold Xin <rx...@databricks.com>.
It's a completely different path.
On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar <lo...@gmail.com> wrote:
> I would like to know if Hive on Spark uses or shares the execution code
> with Spark SQL or DataFrames?
>
> More specifically, does Hive on Spark benefit from the changes made to
> Spark SQL, project Tungsten? Or is it completely different execution path
> where it creates its own plan and executes on RDD?
>
> -Kiran
>
>
Re: Hive on Spark Vs Spark SQL
Posted by Reynold Xin <rx...@databricks.com>.
It's a completely different path.
On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar <lo...@gmail.com> wrote:
> I would like to know if Hive on Spark uses or shares the execution code
> with Spark SQL or DataFrames?
>
> More specifically, does Hive on Spark benefit from the changes made to
> Spark SQL, project Tungsten? Or is it completely different execution path
> where it creates its own plan and executes on RDD?
>
> -Kiran
>
>