You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Santosh Achhra <sa...@gmail.com> on 2013/01/08 10:11:18 UTC
View with map join fails
Hello,
I have created a view as shown below.
*CREATE VIEW V1 AS*
*select /*+ MAPJOIN(t1) ,MAPJOIN(t2) */ t1.f1, t1.f2, t1.f3, t1.f4, t2.f1,
t2.f2, t2.f3 from TABLE1 t1 join TABLE t2 on ( t1.f2= t2.f2 and t1.f3 =
t2.f3 and t1.f4 = t2.f4 ) group by t1.f1, t1.f2, t1.f3, t1.f4, t2.f1,
t2.f2, t2.f3*
View get created successfully however when I execute below mentioned SQL or
any SQL on the view get NULLPOINTER exception error
*hive> select count (*) from V1;*
*FAILED: NullPointerException null*
*hive>*
Is there anything wrong with the view creation ?
Next I created view without MAPJOIN hints
*CREATE VIEW V1 AS*
*select t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3 from TABLE1 t1
join TABLE t2 on ( t1.f2= t2.f2 and t1.f3 = t2.f3 and t1.f4 = t2.f4 ) group
by t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3*
Before executing select SQL I excute *set hive.auto.convert.join=true; *
I am getting beloow mentioned warnings
java.lang.InstantiationException:
org.apache.hadoop.hive.ql.parse.ASTNodeOrigin
Continuing ...
java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
Continuing ...
And I see from log that total 5 mapreduce jobs are
started however when don't set auto.convert.join to true, I see only 3
mapreduce jobs getting invoked.
*Total MapReduce jobs = 5*
*Ended Job = 1116112419, job is filtered out (removed at runtime).*
*Ended Job = -33256989, job is filtered out (removed at runtime).*
*WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please
use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties
files.*
Good wishes,always !
Santosh
Re: View with map join fails
Posted by Santosh Achhra <sa...@gmail.com>.
Thank you Bejoy.
I will file a new Jira.
Good wishes,always !
Santosh
On Wed, Jan 9, 2013 at 2:41 AM, <be...@yahoo.com> wrote:
> **
> Looks like there is a bug with mapjoin + view. Please check hive jira to
> see if there an issue open against this else file a new jira.
>
> From my understanding, When you enable map join, hive parser would create
> back up jobs. These back up jobs are executed only if map join fails. In
> normal cases when map join succeeds these jobs are filtered out and not
> executed.
>
> '1116112419, job is filtered out (removed at runtime).'
>
> Regards
> Bejoy KS
>
> Sent from remote device, Please excuse typos
> ------------------------------
> *From: * Santosh Achhra <sa...@gmail.com>
> *Date: *Tue, 8 Jan 2013 17:11:18 +0800
> *To: *<us...@hive.apache.org>
> *ReplyTo: * user@hive.apache.org
> *Subject: *View with map join fails
>
> Hello,
>
> I have created a view as shown below.
>
> *CREATE VIEW V1 AS*
> *select /*+ MAPJOIN(t1) ,MAPJOIN(t2) */ t1.f1, t1.f2, t1.f3, t1.f4,
> t2.f1, t2.f2, t2.f3 from TABLE1 t1 join TABLE t2 on ( t1.f2= t2.f2 and
> t1.f3 = t2.f3 and t1.f4 = t2.f4 ) group by t1.f1, t1.f2, t1.f3, t1.f4,
> t2.f1, t2.f2, t2.f3*
>
> View get created successfully however when I execute below mentioned SQL
> or any SQL on the view get NULLPOINTER exception error
>
> *hive> select count (*) from V1;*
> *FAILED: NullPointerException null*
> *hive>*
>
> Is there anything wrong with the view creation ?
>
> Next I created view without MAPJOIN hints
>
> *CREATE VIEW V1 AS*
> *select t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3 from TABLE1 t1
> join TABLE t2 on ( t1.f2= t2.f2 and t1.f3 = t2.f3 and t1.f4 = t2.f4 ) group
> by t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3*
>
> Before executing select SQL I excute *set hive.auto.convert.join=true; *
>
> I am getting beloow mentioned warnings
> java.lang.InstantiationException:
> org.apache.hadoop.hive.ql.parse.ASTNodeOrigin
> Continuing ...
> java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
> Continuing ...
>
>
> And I see from log that total 5 mapreduce jobs are
> started however when don't set auto.convert.join to true, I see only 3
> mapreduce jobs getting invoked.
> *Total MapReduce jobs = 5*
> *Ended Job = 1116112419, job is filtered out (removed at runtime).*
> *Ended Job = -33256989, job is filtered out (removed at runtime).*
> *WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated.
> Please use org.apache.hadoop.log.metrics.EventCounter in all the
> log4j.properties files.*
>
>
> Good wishes,always !
> Santosh
>
Re: View with map join fails
Posted by be...@yahoo.com.
Looks like there is a bug with mapjoin + view. Please check hive jira to see if there an issue open against this else file a new jira.
From my understanding, When you enable map join, hive parser would create back up jobs. These back up jobs are executed only if map join fails. In normal cases when map join succeeds these jobs are filtered out and not executed.
'1116112419, job is filtered out (removed at runtime).'
Regards
Bejoy KS
Sent from remote device, Please excuse typos
-----Original Message-----
From: Santosh Achhra <sa...@gmail.com>
Date: Tue, 8 Jan 2013 17:11:18
To: <us...@hive.apache.org>
Reply-To: user@hive.apache.org
Subject: View with map join fails
Hello,
I have created a view as shown below.
*CREATE VIEW V1 AS*
*select /*+ MAPJOIN(t1) ,MAPJOIN(t2) */ t1.f1, t1.f2, t1.f3, t1.f4, t2.f1,
t2.f2, t2.f3 from TABLE1 t1 join TABLE t2 on ( t1.f2= t2.f2 and t1.f3 =
t2.f3 and t1.f4 = t2.f4 ) group by t1.f1, t1.f2, t1.f3, t1.f4, t2.f1,
t2.f2, t2.f3*
View get created successfully however when I execute below mentioned SQL or
any SQL on the view get NULLPOINTER exception error
*hive> select count (*) from V1;*
*FAILED: NullPointerException null*
*hive>*
Is there anything wrong with the view creation ?
Next I created view without MAPJOIN hints
*CREATE VIEW V1 AS*
*select t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3 from TABLE1 t1
join TABLE t2 on ( t1.f2= t2.f2 and t1.f3 = t2.f3 and t1.f4 = t2.f4 ) group
by t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3*
Before executing select SQL I excute *set hive.auto.convert.join=true; *
I am getting beloow mentioned warnings
java.lang.InstantiationException:
org.apache.hadoop.hive.ql.parse.ASTNodeOrigin
Continuing ...
java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
Continuing ...
And I see from log that total 5 mapreduce jobs are
started however when don't set auto.convert.join to true, I see only 3
mapreduce jobs getting invoked.
*Total MapReduce jobs = 5*
*Ended Job = 1116112419, job is filtered out (removed at runtime).*
*Ended Job = -33256989, job is filtered out (removed at runtime).*
*WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please
use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties
files.*
Good wishes,always !
Santosh