You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Bennie Schut <bs...@ebuddy.com> on 2011/03/09 10:39:49 UTC
hive hbase handler metadata NullPointerException
Hi All,
I was trying out hbase 0.89.20100924 with hive trunk with hadoop 0.20.2
When I'm running a simple insert I get this:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 13 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 18 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:335)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
... 18 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:280)
... 30 more
insert overwrite table hbase_table_1 select cldr_id, iso_date from calendar;
I could create the table just fine. like :
CREATE TABLE hbase_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz");
I've set the properties hbase.master, hbase.zookeeper.quorum,
hbase.zookeeper.property.clientPort should that be enough?
Thanks
Bennie.
Re: hive hbase handler metadata NullPointerException
Posted by Bennie Schut <bs...@ebuddy.com>.
I case anyone else runs into this.
I ended up deleting the hbase dir on hdfs installing hbase-0.90.1 and
copying and pasting the hbase configs in the hive-site.xml
Rebuild hive by setting hbase.version to 0.90.1 on
"ivy/libraries.properties"
Then the cli started working but the service still wasn't working so I
suddenly realized jobs started with the service probably didn't have
access to the jars so I run:
add jar /opt/hive/lib/hive-hbase-handler-0.8.0-SNAPSHOT.jar
add jar /opt/hive/lib/hbase-0.90.1-SNAPSHOT.jar
add jar /opt/hive/lib/zookeeper-3.3.1.jar
And then it all started working. This wasn't really evident from the
documentation but in hindsight makes sense. This took a lot more time to
figure out then I'm willing to admit ;-)
Bennie.
On 03/14/2011 12:05 AM, amit jaiswal wrote:
> Hi,
>
> I am also facing the same issue (hive-0.7, hbase-0.90.1, hadoop-0.20.2).
>
> Any help?
>
> -amit
>
> ------------------------------------------------------------------------
> *From:* Bennie Schut <bs...@ebuddy.com>
> *To:* "user@hive.apache.org" <us...@hive.apache.org>
> *Sent:* Wed, 9 March, 2011 4:39:49 AM
> *Subject:* hive hbase handler metadata NullPointerException
>
> Hi All,
>
> I was trying out hbase 0.89.20100924 with hive trunk with hadoop 0.20.2
>
> When I'm running a simple insert I get this:
> java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 5 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> ... 10 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> ... 13 more
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
> ... 18 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:335)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
> at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
> at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
> ... 18 more
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:280)
> ... 30 more
> insert overwrite table hbase_table_1 select cldr_id, iso_date from
> calendar;
>
> I could create the table just fine. like :
> CREATE TABLE hbase_table_1(key int, value string)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
> TBLPROPERTIES ("hbase.table.name" = "xyz");
>
> I've set the properties hbase.master, hbase.zookeeper.quorum,
> hbase.zookeeper.property.clientPort should that be enough?
>
> Thanks
> Bennie.
Re: hive hbase handler metadata NullPointerException
Posted by amit jaiswal <am...@yahoo.com>.
Hi,
I am also facing the same issue (hive-0.7, hbase-0.90.1, hadoop-0.20.2).
Any help?
-amit
________________________________
From: Bennie Schut <bs...@ebuddy.com>
To: "user@hive.apache.org" <us...@hive.apache.org>
Sent: Wed, 9 March, 2011 4:39:49 AM
Subject: hive hbase handler metadata NullPointerException
Hi All,
I was trying out hbase 0.89.20100924 with hive trunk with hadoop 0.20.2
When I'm running a simple insert I get this:
java.lang.RuntimeException: Error in configuring object at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at
org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by:
java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ...
5 more Caused by: java.lang.RuntimeException: Error in configuring object at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at
org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) ... 10 more
Caused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ...
13 more Caused by: java.lang.RuntimeException: Map operator initialization
failed at
org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121) ... 18
more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:335)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433) at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433) at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357) at
org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98) ... 18
more Caused by: java.lang.NullPointerException at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:280)
... 30 moreinsert overwrite table hbase_table_1 select cldr_id, iso_date
from calendar;
I could create the table just fine. like :
CREATE TABLE hbase_table_1(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val")
TBLPROPERTIES ("hbase.table.name" = "xyz");
I've set the properties hbase.master, hbase.zookeeper.quorum,
hbase.zookeeper.property.clientPort should that be enough?
Thanks
Bennie.