You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Larry Ogrodnek <la...@bizo.com> on 2009/07/01 01:43:36 UTC
NullPointerException when inserting into a partition with a select
from another table/partition
trying to run the following query:
insert overwrite table imp_test3 partition(ds="20090513") select
d["query_string"] from imp where imp.ds="20090513" and imp.hour="00";
causes all of the tasks in step 2 to fail with:
java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:134)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:175)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:203)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:264)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:94)
... 3 more
imp_test3 is defined as:
create table imp_test3(qs string)
partitioned by (ds string)
stored as sequencefile
;
If I get rid of the partition in the destination table, and use:
insert overwrite table imp_test3 select d["query_string"] from imp where
imp.ds="20090513" and imp.hour="00";
then everything seems to work....
I also tried adding another partition to the destination table (hour), to
match the input... but, still no go...
imp is defined as:
CREATE TABLE imp (d MAP<STRING,STRING>)
partitioned by (ds string, hour string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
COLLECTION ITEMS TERMINATED BY '\1'
MAP KEYS TERMINATED BY '\2'
STORED AS TEXTFILE
;
thanks for the help.
Re: NullPointerException when inserting into a partition with a
select from another table/partition
Posted by Larry Ogrodnek <la...@bizo.com>.
Zheng,
Thanks for the help. I tried both attaching a file with the output and
including it inline, but it keeps getting rejected by the listserver as
spam.
I thought I would try giving you an external link to the output:
http://com-bizo-public.s3.amazonaws.com/hive/debug/partition_imp3.txt
Hope this works...
thanks,
larry
On Tue, Jun 30, 2009 at 10:38 PM, Zheng Shao <zs...@gmail.com> wrote:
> Can you do the following command and send us the result?
>
> explain extended insert overwrite table imp_test3
> partition(ds="20090513") select d["query_string"] from imp where
> imp.ds="20090513" and imp.hour="00";
>
> Zheng
>
> On Tue, Jun 30, 2009 at 4:43 PM, Larry Ogrodnek<la...@bizo.com> wrote:
> > trying to run the following query:
> >
> > insert overwrite table imp_test3 partition(ds="20090513") select
> > d["query_string"] from imp where imp.ds="20090513" and imp.hour="00";
> >
> > causes all of the tasks in step 2 to fail with:
> >
> > java.lang.RuntimeException: Map operator initialization failed
> > at
> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:134)
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> >
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> > at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
> > Caused by: java.lang.NullPointerException
> > at
> >
> org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:175)
> >
> >
> > at
> >
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:203)
> > at
> org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:264)
> > at
> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:94)
> >
> >
> > ... 3 more
> >
> > imp_test3 is defined as:
> > create table imp_test3(qs string)
> > partitioned by (ds string)
> > stored as sequencefile
> > ;
> >
> > If I get rid of the partition in the destination table, and use:
> > insert overwrite table imp_test3 select d["query_string"] from imp where
> > imp.ds="20090513" and imp.hour="00";
> >
> > then everything seems to work....
> >
> > I also tried adding another partition to the destination table (hour), to
> > match the input... but, still no go...
> >
> > imp is defined as:
> > CREATE TABLE imp (d MAP<STRING,STRING>)
> > partitioned by (ds string, hour string)
> > ROW FORMAT DELIMITED
> > FIELDS TERMINATED BY '\t'
> > COLLECTION ITEMS TERMINATED BY '\1'
> > MAP KEYS TERMINATED BY '\2'
> > STORED AS TEXTFILE
> > ;
> >
> > thanks for the help.
> >
> >
>
>
>
> --
> Yours,
> Zheng
>
Re: NullPointerException when inserting into a partition with a
select from another table/partition
Posted by Zheng Shao <zs...@gmail.com>.
Can you do the following command and send us the result?
explain extended insert overwrite table imp_test3
partition(ds="20090513") select d["query_string"] from imp where
imp.ds="20090513" and imp.hour="00";
Zheng
On Tue, Jun 30, 2009 at 4:43 PM, Larry Ogrodnek<la...@bizo.com> wrote:
> trying to run the following query:
>
> insert overwrite table imp_test3 partition(ds="20090513") select
> d["query_string"] from imp where imp.ds="20090513" and imp.hour="00";
>
> causes all of the tasks in step 2 to fail with:
>
> java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:134)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>
>
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:175)
>
>
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:203)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:264)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:94)
>
>
> ... 3 more
>
> imp_test3 is defined as:
> create table imp_test3(qs string)
> partitioned by (ds string)
> stored as sequencefile
> ;
>
> If I get rid of the partition in the destination table, and use:
> insert overwrite table imp_test3 select d["query_string"] from imp where
> imp.ds="20090513" and imp.hour="00";
>
> then everything seems to work....
>
> I also tried adding another partition to the destination table (hour), to
> match the input... but, still no go...
>
> imp is defined as:
> CREATE TABLE imp (d MAP<STRING,STRING>)
> partitioned by (ds string, hour string)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t'
> COLLECTION ITEMS TERMINATED BY '\1'
> MAP KEYS TERMINATED BY '\2'
> STORED AS TEXTFILE
> ;
>
> thanks for the help.
>
>
--
Yours,
Zheng