You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by pseudo oduesp <ps...@gmail.com> on 2016/05/31 03:26:13 UTC
equvalent beewn join sql and data frame
hi guys ,
it s similare thing to do :
sqlcontext.join("select * from t1 join t2 on condition) and
df1.join(df2,condition,'inner")??????
ps: df1.registertable('t1')
ps: df2.registertable('t2')
thanks
Re: equvalent beewn join sql and data frame
Posted by Mich Talebzadeh <mi...@gmail.com>.
one is sql and the other one is its equivalent in functional programming
val s =
HiveContext.table("sales").select("AMOUNT_SOLD","TIME_ID","CHANNEL_ID")
val c = HiveContext.table("channels").select("CHANNEL_ID","CHANNEL_DESC")
val t = HiveContext.table("times").select("TIME_ID","CALENDAR_MONTH_DESC")
//These are the same
val rs =
s.join(t,s("time_id")===t("time_id")).join(c,"channel_id").groupBy("calendar_month_desc","channel_desc").agg(sum("amount_sold").as("TotalSales"))
val rs =
s.join(t,"time_id").join(c,"channel_id").groupBy("calendar_month_desc","channel_desc").agg(sum("amount_sold").as("TotalSales"))
HTH
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 31 May 2016 at 04:55, Takeshi Yamamuro <li...@gmail.com> wrote:
> Hi,
>
> The same they are.
> If you check the equality, you can use DataFrame#explain.
>
> // maropu
>
>
> On Tue, May 31, 2016 at 12:26 PM, pseudo oduesp <ps...@gmail.com>
> wrote:
>
>> hi guys ,
>> it s similare thing to do :
>>
>> sqlcontext.join("select * from t1 join t2 on condition) and
>>
>> df1.join(df2,condition,'inner")??????
>>
>> ps: df1.registertable('t1')
>> ps: df2.registertable('t2')
>> thanks
>>
>
>
>
> --
> ---
> Takeshi Yamamuro
>
Re: equvalent beewn join sql and data frame
Posted by Takeshi Yamamuro <li...@gmail.com>.
Hi,
The same they are.
If you check the equality, you can use DataFrame#explain.
// maropu
On Tue, May 31, 2016 at 12:26 PM, pseudo oduesp <ps...@gmail.com>
wrote:
> hi guys ,
> it s similare thing to do :
>
> sqlcontext.join("select * from t1 join t2 on condition) and
>
> df1.join(df2,condition,'inner")??????
>
> ps: df1.registertable('t1')
> ps: df2.registertable('t2')
> thanks
>
--
---
Takeshi Yamamuro