You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by abhiTowson cal <ab...@gmail.com> on 2012/07/26 21:20:37 UTC
STREAM (TABLE) IN HIVE
HI all,
Can any one explain me how stream table in hive works
What is the importance of STREAM TABLE?
When should i use it?
Suppose i am joining four to five tables
Which table should be mentioned in the STREAM table hint?
Should Big Table be specified in the STREAM TABLE hint or Small Table
Regards
Abhishek
Re: STREAM (TABLE) IN HIVE
Posted by abhiTowson cal <ab...@gmail.com>.
Thank you Bejoy.
Regards
Abhishek
On Thu, Jul 26, 2012 at 3:57 PM, Bejoy Ks <be...@yahoo.com> wrote:
> Hi Abshiek
>
> From my understanding, when you have the join happening in map or reduce,
> the values corresponding to a key from all all table's except one (if two
> tables are involved in join on same key, then just one table here) are
> buffered in memory and the left out one is streamed. Usually it is the
> largest table to be streamed, else the larger data can go into the
> memory(buffer) and create OOM errors.
>
> This stream table hint is used to specify which table to be streamed. By
> default it is the table that comes on the right is streamed and the other is
> buffered. But if you wan't other than right table to be streamed you go
> for this hint.
>
> If you are joining more tables on different keys, then for every join set
> just specify the larger table on the right of ON condition. No need of
> stream table hint here.
>
>
> Regards
> Bejoy KS
>
> ________________________________
> From: abhiTowson cal <ab...@gmail.com>
> To: user@hive.apache.org
> Sent: Friday, July 27, 2012 12:50 AM
> Subject: STREAM (TABLE) IN HIVE
>
> HI all,
>
> Can any one explain me how stream table in hive works
> What is the importance of STREAM TABLE?
> When should i use it?
> Suppose i am joining four to five tables
> Which table should be mentioned in the STREAM table hint?
> Should Big Table be specified in the STREAM TABLE hint or Small Table
>
> Regards
> Abhishek
>
>
Re: STREAM (TABLE) IN HIVE
Posted by Bejoy Ks <be...@yahoo.com>.
Hi Abshiek
From my understanding, when you have the join happening in map or reduce, the values corresponding to a key from all all table's except one (if two tables are involved in join on same key, then just one table here) are buffered in memory and the left out one is streamed. Usually it is the largest table to be streamed, else the larger data can go into the memory(buffer) and create OOM errors.
This stream table hint is used to specify which table to be streamed. By default it is the table that comes on the right is streamed and the other is buffered. But if you wan't other than right table to be streamed you go for this hint.
If you are joining more tables on different keys, then for every join set just specify the larger table on the right of ON condition. No need of stream table hint here.
Regards
Bejoy KS
________________________________
From: abhiTowson cal <ab...@gmail.com>
To: user@hive.apache.org
Sent: Friday, July 27, 2012 12:50 AM
Subject: STREAM (TABLE) IN HIVE
HI all,
Can any one explain me how stream table in hive works
What is the importance of STREAM TABLE?
When should i use it?
Suppose i am joining four to five tables
Which table should be mentioned in the STREAM table hint?
Should Big Table be specified in the STREAM TABLE hint or Small Table
Regards
Abhishek