You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by kant kodali <ka...@gmail.com> on 2017/07/07 00:28:31 UTC

If I pass raw SQL string to dataframe do I still get the Spark SQL optimizations?

HI All,

I am wondering If I pass a raw SQL string to dataframe do I still get the
Spark SQL optimizations? why or why not?

Thanks!

Re: If I pass raw SQL string to dataframe do I still get the Spark SQL optimizations?

Posted by ayan guha <gu...@gmail.com>.

I kind of think "Thats the whole point" :) Sorry it is Friday here :) :)

On Fri, Jul 7, 2017 at 1:09 PM, Michael Armbrust <mi...@databricks.com>
wrote:

> It goes through the same optimization pipeline.  More in this video
> <https://youtu.be/1a4pgYzeFwE?t=608>.
>
> On Thu, Jul 6, 2017 at 5:28 PM, kant kodali <ka...@gmail.com> wrote:
>
>> HI All,
>>
>> I am wondering If I pass a raw SQL string to dataframe do I still get the
>> Spark SQL optimizations? why or why not?
>>
>> Thanks!
>>
>
>


-- 
Best Regards,
Ayan Guha

Re: If I pass raw SQL string to dataframe do I still get the Spark SQL optimizations?

Posted by Michael Armbrust <mi...@databricks.com>.

It goes through the same optimization pipeline.  More in this video
<https://youtu.be/1a4pgYzeFwE?t=608>.

On Thu, Jul 6, 2017 at 5:28 PM, kant kodali <ka...@gmail.com> wrote:

> HI All,
>
> I am wondering If I pass a raw SQL string to dataframe do I still get the
> Spark SQL optimizations? why or why not?
>
> Thanks!
>