You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ruben de Vries <ru...@hyves.nl> on 2012/05/01 09:30:03 UTC
RE: JOIN + LATERAL VIEW + MAPJOIN = no output?!
I really do feel like this isn't as intended, should I make a ticket in JIRA?
-----Original Message-----
From: Ruben de Vries [mailto:ruben.devries@hyves.nl]
Sent: Thursday, April 26, 2012 3:37 PM
To: user@hive.apache.org
Subject: RE: JOIN + LATERAL VIEW + MAPJOIN = no output?!
https://gist.github.com/2499658
and this is the plan.xml its using
-----Original Message-----
From: Ruben de Vries [mailto:ruben.devries@hyves.nl]
Sent: Thursday, April 26, 2012 3:17 PM
To: user@hive.apache.org
Subject: JOIN + LATERAL VIEW + MAPJOIN = no output?!
Okay first off; so JOIN + LATERAL VIEW together isn't working so I moved my JOIN into a subquery and that makes the query work properly
However when I added a MAPJOIN hint for the JOIN in the subquery it will also stop doing the reducer for the main query!
This only happens when there's a LATERAL VIEW in there though, if I remove the LATERAL VIEW then the main query still get's a reducer to do grouping
Here's a gist: https://gist.github.com/2499436 Containing the queries and a PHP script which you can run to execute the test case;
- setup a database called hive_mapjoin
- setup tables
- load some test data
- do the selects
You'll need the https://github.com/rcongiu/Hive-JSON-Serde/downloads json-serde-1.1-jar-with-dependencies.jar with it though and change the path ;)
Not sure if this is a bug or me doing something that just isn’t supposed to be working,
Thanks Ruben
Re: JOIN + LATERAL VIEW + MAPJOIN = no output?!
Posted by Edward Capriolo <ed...@gmail.com>.
Yes make a ticket
On Tuesday, May 1, 2012, Ruben de Vries <ru...@hyves.nl> wrote:
> I really do feel like this isn't as intended, should I make a ticket in
JIRA?
>
> -----Original Message-----
> From: Ruben de Vries [mailto:ruben.devries@hyves.nl]
> Sent: Thursday, April 26, 2012 3:37 PM
> To: user@hive.apache.org
> Subject: RE: JOIN + LATERAL VIEW + MAPJOIN = no output?!
>
> https://gist.github.com/2499658
>
> and this is the plan.xml its using
>
> -----Original Message-----
> From: Ruben de Vries [mailto:ruben.devries@hyves.nl]
> Sent: Thursday, April 26, 2012 3:17 PM
> To: user@hive.apache.org
> Subject: JOIN + LATERAL VIEW + MAPJOIN = no output?!
>
> Okay first off; so JOIN + LATERAL VIEW together isn't working so I moved
my JOIN into a subquery and that makes the query work properly
>
> However when I added a MAPJOIN hint for the JOIN in the subquery it will
also stop doing the reducer for the main query!
> This only happens when there's a LATERAL VIEW in there though, if I
remove the LATERAL VIEW then the main query still get's a reducer to do
grouping
>
> Here's a gist: https://gist.github.com/2499436 Containing the queries and
a PHP script which you can run to execute the test case;
> - setup a database called hive_mapjoin
> - setup tables
> - load some test data
> - do the selects
> You'll need the https://github.com/rcongiu/Hive-JSON-Serde/downloadsjson-serde-1.1-jar-with-dependencies.jar with it though and change the path
;)
>
> Not sure if this is a bug or me doing something that just isn’t supposed
to be working,
>
> Thanks Ruben
>