You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ruben de Vries <ru...@hyves.nl> on 2012/05/01 09:30:03 UTC

RE: JOIN + LATERAL VIEW + MAPJOIN = no output?!

I really do feel like this isn't as intended, should I make a ticket in JIRA?

-----Original Message-----
From: Ruben de Vries [mailto:ruben.devries@hyves.nl] 
Sent: Thursday, April 26, 2012 3:37 PM
To: user@hive.apache.org
Subject: RE: JOIN + LATERAL VIEW + MAPJOIN = no output?!

https://gist.github.com/2499658

and this is the plan.xml its using

-----Original Message-----
From: Ruben de Vries [mailto:ruben.devries@hyves.nl] 
Sent: Thursday, April 26, 2012 3:17 PM
To: user@hive.apache.org
Subject: JOIN + LATERAL VIEW + MAPJOIN = no output?!

Okay first off; so JOIN + LATERAL VIEW together isn't working so I moved my JOIN into a subquery and that makes the query work properly

However when I added a MAPJOIN hint for the JOIN in the subquery it will also stop doing the reducer for the main query!
This only happens when there's a LATERAL VIEW in there though, if I remove the LATERAL VIEW then the main query still get's a reducer to do grouping

Here's a gist: https://gist.github.com/2499436 Containing the queries and a PHP script which you can run to execute the test case;
 - setup a database called hive_mapjoin
 - setup tables
 - load some test data
 - do the selects
You'll need the https://github.com/rcongiu/Hive-JSON-Serde/downloads json-serde-1.1-jar-with-dependencies.jar with it though and change the path ;)

Not sure if this is a bug or me doing something that just isn’t supposed to be working,

Thanks Ruben

Re: JOIN + LATERAL VIEW + MAPJOIN = no output?!

Posted by Edward Capriolo <ed...@gmail.com>.
Yes make a ticket

On Tuesday, May 1, 2012, Ruben de Vries <ru...@hyves.nl> wrote:
> I really do feel like this isn't as intended, should I make a ticket in
JIRA?
>
> -----Original Message-----
> From: Ruben de Vries [mailto:ruben.devries@hyves.nl]
> Sent: Thursday, April 26, 2012 3:37 PM
> To: user@hive.apache.org
> Subject: RE: JOIN + LATERAL VIEW + MAPJOIN = no output?!
>
> https://gist.github.com/2499658
>
> and this is the plan.xml its using
>
> -----Original Message-----
> From: Ruben de Vries [mailto:ruben.devries@hyves.nl]
> Sent: Thursday, April 26, 2012 3:17 PM
> To: user@hive.apache.org
> Subject: JOIN + LATERAL VIEW + MAPJOIN = no output?!
>
> Okay first off; so JOIN + LATERAL VIEW together isn't working so I moved
my JOIN into a subquery and that makes the query work properly
>
> However when I added a MAPJOIN hint for the JOIN in the subquery it will
also stop doing the reducer for the main query!
> This only happens when there's a LATERAL VIEW in there though, if I
remove the LATERAL VIEW then the main query still get's a reducer to do
grouping
>
> Here's a gist: https://gist.github.com/2499436 Containing the queries and
a PHP script which you can run to execute the test case;
>  - setup a database called hive_mapjoin
>  - setup tables
>  - load some test data
>  - do the selects
> You'll need the https://github.com/rcongiu/Hive-JSON-Serde/downloadsjson-serde-1.1-jar-with-dependencies.jar with it though and change the path
;)
>
> Not sure if this is a bug or me doing something that just isn’t supposed
to be working,
>
> Thanks Ruben
>