You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Julio Lopez <jc...@andrew.cmu.edu> on 2011/04/23 22:25:54 UTC
Help with NPE during bulk load (completebulkload)
Folks, some help would be greatly appreciated.
I'm getting the following NPE while trying a bulk load with the completebulkload tool (see command below). The error is in the zookeeper code that reads the zk config. The possible NPs seem to be:
- conf
- value
The following is in hbase-site.xml
---------------
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
---------------
The hbase shell works fine, I can connect to HBase and get description of the tables.
The zoo.cfg file is also included.
Has anybody experienced a similar issue?
Thanks in advance,
-Julio
----- command ----
HADOOP_CLASSPATH=${HBASE_CONF_DIR} hadoop --config ${HADOOP_CONF_DIR} \
jar /usr/local/sw/hbase/hbase-0.90.2.jar completebulkload \
bulkgen/output/files usertable
--- code snippet from org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167) ---
// Special case for 'hbase.cluster.distributed' property being 'true'
if (key.startsWith("server.")) {
167: ====> if (conf.get(HConstants.CLUSTER_DISTRIBUTED).equals(HConstants.CLUSTER_IS_DISTRIBUTED)
&& value.startsWith("localhost")) {
---------------------
/usr/local/sw/hadoop/bin/hadoopException in thread "main" java.lang.NullPointerException
at org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167)
at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:69)
at org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:250)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:113)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:998)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:303)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:294)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:156)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:167)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:311)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:318)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
-------------- zoo.cfg --------------
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zk/data
dataLogDir=/var/lib/zk/log
# the port at which the clients will connect
clientPort=2181
#server.1=zk1:2888:3888
server.2=zk2:2888:3888
server.3=zk3:2888:3888
server.4=zk4:2888:3888
----------------------------------------
RE: Help with NPE during bulk load (completebulkload)
Posted by Andy Sautins <an...@returnpath.net>.
What version of HBase are you running? We had a similar issue and were able to work around it by passing -Dhbase.cluster.distributed=true on the command line to completebulkload.
-----Original Message-----
From: Julio Lopez [mailto:jclopez@andrew.cmu.edu]
Sent: Saturday, April 23, 2011 2:26 PM
To: user@hbase.apache.org
Subject: Help with NPE during bulk load (completebulkload)
Folks, some help would be greatly appreciated.
I'm getting the following NPE while trying a bulk load with the completebulkload tool (see command below). The error is in the zookeeper code that reads the zk config. The possible NPs seem to be:
- conf
- value
The following is in hbase-site.xml
---------------
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
---------------
The hbase shell works fine, I can connect to HBase and get description of the tables.
The zoo.cfg file is also included.
Has anybody experienced a similar issue?
Thanks in advance,
-Julio
----- command ----
HADOOP_CLASSPATH=${HBASE_CONF_DIR} hadoop --config ${HADOOP_CONF_DIR} \
jar /usr/local/sw/hbase/hbase-0.90.2.jar completebulkload \
bulkgen/output/files usertable
--- code snippet from org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167) ---
// Special case for 'hbase.cluster.distributed' property being 'true'
if (key.startsWith("server.")) {
167: ====> if (conf.get(HConstants.CLUSTER_DISTRIBUTED).equals(HConstants.CLUSTER_IS_DISTRIBUTED)
&& value.startsWith("localhost")) {
---------------------
/usr/local/sw/hadoop/bin/hadoopException in thread "main" java.lang.NullPointerException
at org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167)
at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:69)
at org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:250)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:113)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:998)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:303)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:294)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:156)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:167)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:311)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:318)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
-------------- zoo.cfg --------------
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between # sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zk/data
dataLogDir=/var/lib/zk/log
# the port at which the clients will connect
clientPort=2181
#server.1=zk1:2888:3888
server.2=zk2:2888:3888
server.3=zk3:2888:3888
server.4=zk4:2888:3888
----------------------------------------