You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Michael Seibold <se...@in.tum.de> on 2009/01/19 10:13:05 UTC
Problem with HBase and HDFS
Hi,
I get the following error when using HBase with HDFS. When I configure
HBase to write to the local file system everything works fine.
I'm using:
hadoop-0.18.2
hbase-0.18.1
I have installed hadoop on node1:
- hadoop-site.xml:
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://node1/</value>
</property>
</configuration>
-slaves:
node1
I have installed hbase on node2:
- hbase-site.xml:
<configuration>
<property>
<name>hbase.master</name>
<value>node2:9000</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://node1:8020/hbase</value>
</property>
</configuration>
- regionservers:
node2
I format the hadoop namenode: $HADOOP_HOME/bin/hadoop namenode -format
I start hadoop on node1: $HADOOP_HOME/bin/start-dfs.sh
(I can use the browser interface to browse the HDFS, HDFS seems to work
fine)
I start hbase on node2: $HBASE_HOME/bin/start-hbase.sh
I start the hbase shell on node2: $HBASE_HOME/bin/hbase shell, enter the
command "list" in order to list the existing tables and get the
following NullPointerException:
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.18.1, r707159, Wed Oct 22 12:43:06 PDT 2008
hbase(main):001:0> list
NativeException: java.lang.NullPointerException: null
from org/apache/hadoop/hbase/client/ServerCallable.java:71:in
`getRegionName'
from
org/apache/hadoop/hbase/client/HConnectionManager.java:863:in
`getRegionServerWithRetries'
from org/apache/hadoop/hbase/client/MetaScanner.java:56:in
`metaScan'
from org/apache/hadoop/hbase/client/MetaScanner.java:30:in
`metaScan'
from
org/apache/hadoop/hbase/client/HConnectionManager.java:297:in
`listTables'
from org/apache/hadoop/hbase/client/HBaseAdmin.java:117:in
`listTables'
from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
from sun/reflect/DelegatingMethodAccessorImpl.java:25:in
`invoke'
from java/lang/reflect/Method.java:585:in `invoke'
from org/jruby/javasupport/JavaMethod.java:250:in
`invokeWithExceptionHandling'
from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
from org/jruby/javasupport/JavaClass.java:416:in `execute'
from
org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in
`call'
from org/jruby/internal/runtime/methods/DynamicMethod.java:70:in
`call'
from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
... 131 levels...
from
ruby.home.ntap.seibold.distributed_hbase.hbase_minus_0_dot_18_dot_1.bin.hirbInvokermethod__23$RUBY$startOpt:-1:in `call'
from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in
`call'
from
org/jruby/internal/runtime/methods/CompiledMethod.java:48:in
`call'
from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
from org/jruby/runtime/CallSite.java:298:in `call'
from
ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//home/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:351:in `__file__'
from
ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//home/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:-1:in `__file__'
from
ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//home/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:-1:in `load'
from org/jruby/Ruby.java:512:in `runScript'
from org/jruby/Ruby.java:432:in `runNormally'
from org/jruby/Ruby.java:312:in `runFromMain'
from org/jruby/Main.java:144:in `run'
from org/jruby/Main.java:89:in `run'
from org/jruby/Main.java:80:in `main'
from /home/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:242:in `list'
from (hbase):2:in `binding'hbase(main):002:0>
(In the browser interface of HDFS I can see, that the directory hbase
has been created and contains some folders and files)
What am I doing wrong?
Kind regards,
Michael
Re: Problem with HBase and HDFS
Posted by Rong-en Fan <gr...@gmail.com>.
On Mon, Jan 19, 2009 at 5:55 PM, Michael Seibold <se...@in.tum.de> wrote:
> Hi Gennady,
>
> Thanks for your response. Yes I looked into the log file of the region
> server and everything looks fine until I execute the command in the
> hbase shell. At this point the exception is shown in the hbase shell and
> also in the log file of the region server.
>
> The method getRegionName() in class ServerCallable throws an
> NullPointerException, because the call location.getRegionInfo() returns
> null. The questions is what's wrong in my setup to cause this.
>
> I compared the insatllation/configuration steps I did with the steps in
> your guide and the only difference that I can find is that you use Cent
> OS (I was using Ubuntu) and you disabled IPv6 (I didn't do that, as far
> as I remember that was not mentioned in th official install guides).
>
> Why did you disable IPv6? Is this a required step?
I guess it's because the interaction between jvm and ipv6, see
http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200812.mbox/%3CC56ED656.C28C%25runping@yahoo-inc.com%3E
Hope this helps,
Rong-En Fan
RE: Problem with HBase and HDFS
Posted by Michael Seibold <se...@in.tum.de>.
Hi,
on a single node installation it worked now with HDFS after disabling
ipv6 and after changing the ports. Next I will try it on separate
machines.
Thanks a lot,
Michael
El lun, 19-01-2009 a las 03:36 -0800, Jean-Adrien escribió:
> Hi,
>
> Your configuration seems to invert usual ports used.
> In hadoop-site file you define the listening port of you master
> <name>fs.default.name</name>
> <value>hdfs://node5-2:9000</value>
>
> in your case the port is missing. Perhaps it causes problems
> In the hbase-site file, you have to use the same port in order to connect to
> hadoop dfs:
>
> <name>hbase.rootdir</name>
> <value>hdfs://node5-2:9000/hbase</value>
>
> In your case you have an non default value of 8020. And you use 9000 as your
> listening port for HBase, which is, by default 60000 :
> <value>node5-2:60000</value>
> <description>The host and port that the HBase master runs at.
>
> In my case HBase master and Hadoop nameserver runs on the same machine
> (node5-2)
> By the way, if you read the tutorial 'Running Hadoop On Ubuntu Linux
> (Single-Node Cluster)'
> http://wiki.apache.org/hadoop/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)
> notice that it proposes non-default ports for hadoop config. Usual listening
> port are more likely
> 9000 for dfs (as we seen) and 9001 for map-reduce job tracker
>
> Hope it solves the problem.
>
> -- Jean-Adrien
>
>
> Michael Seibold wrote:
> >
> > Hi Gennady,
> >
> > Thanks for your response. Yes I looked into the log file of the region
> > server and everything looks fine until I execute the command in the
> > hbase shell. At this point the exception is shown in the hbase shell and
> > also in the log file of the region server.
> >
> > The method getRegionName() in class ServerCallable throws an
> > NullPointerException, because the call location.getRegionInfo() returns
> > null. The questions is what's wrong in my setup to cause this.
> >
> > I compared the insatllation/configuration steps I did with the steps in
> > your guide and the only difference that I can find is that you use Cent
> > OS (I was using Ubuntu) and you disabled IPv6 (I didn't do that, as far
> > as I remember that was not mentioned in th official install guides).
> >
> > Why did you disable IPv6? Is this a required step?
> >
> > Thanks,
> > Michael
> >
> >
> >
>
RE: Problem with HBase and HDFS
Posted by Jean-Adrien <ad...@jeanjean.ch>.
Hi,
Your configuration seems to invert usual ports used.
In hadoop-site file you define the listening port of you master
<name>fs.default.name</name>
<value>hdfs://node5-2:9000</value>
in your case the port is missing. Perhaps it causes problems
In the hbase-site file, you have to use the same port in order to connect to
hadoop dfs:
<name>hbase.rootdir</name>
<value>hdfs://node5-2:9000/hbase</value>
In your case you have an non default value of 8020. And you use 9000 as your
listening port for HBase, which is, by default 60000 :
<value>node5-2:60000</value>
<description>The host and port that the HBase master runs at.
In my case HBase master and Hadoop nameserver runs on the same machine
(node5-2)
By the way, if you read the tutorial 'Running Hadoop On Ubuntu Linux
(Single-Node Cluster)'
http://wiki.apache.org/hadoop/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)
notice that it proposes non-default ports for hadoop config. Usual listening
port are more likely
9000 for dfs (as we seen) and 9001 for map-reduce job tracker
Hope it solves the problem.
-- Jean-Adrien
Michael Seibold wrote:
>
> Hi Gennady,
>
> Thanks for your response. Yes I looked into the log file of the region
> server and everything looks fine until I execute the command in the
> hbase shell. At this point the exception is shown in the hbase shell and
> also in the log file of the region server.
>
> The method getRegionName() in class ServerCallable throws an
> NullPointerException, because the call location.getRegionInfo() returns
> null. The questions is what's wrong in my setup to cause this.
>
> I compared the insatllation/configuration steps I did with the steps in
> your guide and the only difference that I can find is that you use Cent
> OS (I was using Ubuntu) and you disabled IPv6 (I didn't do that, as far
> as I remember that was not mentioned in th official install guides).
>
> Why did you disable IPv6? Is this a required step?
>
> Thanks,
> Michael
>
>
>
--
View this message in context: http://www.nabble.com/Problem-with-HBase-and-HDFS-tp21539226p21541382.html
Sent from the HBase User mailing list archive at Nabble.com.
RE: Problem with HBase and HDFS
Posted by Michael Seibold <se...@in.tum.de>.
Hi Gennady,
Thanks for your response. Yes I looked into the log file of the region
server and everything looks fine until I execute the command in the
hbase shell. At this point the exception is shown in the hbase shell and
also in the log file of the region server.
The method getRegionName() in class ServerCallable throws an
NullPointerException, because the call location.getRegionInfo() returns
null. The questions is what's wrong in my setup to cause this.
I compared the insatllation/configuration steps I did with the steps in
your guide and the only difference that I can find is that you use Cent
OS (I was using Ubuntu) and you disabled IPv6 (I didn't do that, as far
as I remember that was not mentioned in th official install guides).
Why did you disable IPv6? Is this a required step?
Thanks,
Michael
El lun, 19-01-2009 a las 11:32 +0200, Genady escribió:
> Michael,
>
> Did you looked at region server log at $HBASE_HOME/logs? Looks like
> configuration issue.
> You could look on example of single node configuration in my blog:
>
> http://javadialog.blogspot.com/2008/12/running-hadoop-with-hbase-on-centos.h
> tml
>
>
> Gennady
>
>
> -----Original Message-----
> From: Michael Seibold [mailto:seibold@in.tum.de]
> Sent: Monday, January 19, 2009 11:13 AM
> To: hbase-user@hadoop.apache.org
> Subject: Problem with HBase and HDFS
>
> Hi,
>
> I get the following error when using HBase with HDFS. When I configure
> HBase to write to the local file system everything works fine.
>
> I'm using:
> hadoop-0.18.2
> hbase-0.18.1
>
> I have installed hadoop on node1:
> - hadoop-site.xml:
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://node1/</value>
> </property>
> </configuration>
> -slaves:
> node1
>
> I have installed hbase on node2:
> - hbase-site.xml:
> <configuration>
> <property>
> <name>hbase.master</name>
> <value>node2:9000</value>
> </property>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://node1:8020/hbase</value>
> </property>
> </configuration>
> - regionservers:
> node2
>
> I format the hadoop namenode: $HADOOP_HOME/bin/hadoop namenode -format
>
> I start hadoop on node1: $HADOOP_HOME/bin/start-dfs.sh
>
> (I can use the browser interface to browse the HDFS, HDFS seems to work
> fine)
>
> I start hbase on node2: $HBASE_HOME/bin/start-hbase.sh
>
> I start the hbase shell on node2: $HBASE_HOME/bin/hbase shell, enter the
> command "list" in order to list the existing tables and get the
> following NullPointerException:
>
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Version: 0.18.1, r707159, Wed Oct 22 12:43:06 PDT 2008
> hbase(main):001:0> list
> NativeException: java.lang.NullPointerException: null
> from org/apache/hadoop/hbase/client/ServerCallable.java:71:in
> `getRegionName'
> from
> org/apache/hadoop/hbase/client/HConnectionManager.java:863:in
> `getRegionServerWithRetries'
> from org/apache/hadoop/hbase/client/MetaScanner.java:56:in
> `metaScan'
> from org/apache/hadoop/hbase/client/MetaScanner.java:30:in
> `metaScan'
> from
> org/apache/hadoop/hbase/client/HConnectionManager.java:297:in
> `listTables'
> from org/apache/hadoop/hbase/client/HBaseAdmin.java:117:in
> `listTables'
> from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
> from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
> from sun/reflect/DelegatingMethodAccessorImpl.java:25:in
> `invoke'
> from java/lang/reflect/Method.java:585:in `invoke'
> from org/jruby/javasupport/JavaMethod.java:250:in
> `invokeWithExceptionHandling'
> from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
> from org/jruby/javasupport/JavaClass.java:416:in `execute'
> from
> org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in
> `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:70:in
> `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> ... 131 levels...
> from
> ruby.home.ntap.seibold.distributed_hbase.hbase_minus_0_dot_18_dot_1.bin.hirb
> Invokermethod__23$RUBY$startOpt:-1:in `call'
> from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in
> `call'
> from
> org/jruby/internal/runtime/methods/CompiledMethod.java:48:in
> `call'
> from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
> from org/jruby/runtime/CallSite.java:298:in `call'
> from
> ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//hom
> e/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:351:in
> `__file__'
> from
> ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//hom
> e/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:-1:in
> `__file__'
> from
> ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//hom
> e/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:-1:in
> `load'
> from org/jruby/Ruby.java:512:in `runScript'
> from org/jruby/Ruby.java:432:in `runNormally'
> from org/jruby/Ruby.java:312:in `runFromMain'
> from org/jruby/Main.java:144:in `run'
> from org/jruby/Main.java:89:in `run'
> from org/jruby/Main.java:80:in `main'
>
> from
> /home/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:242:in
> `list'
> from (hbase):2:in `binding'hbase(main):002:0>
>
> (In the browser interface of HDFS I can see, that the directory hbase
> has been created and contains some folders and files)
>
> What am I doing wrong?
>
> Kind regards,
> Michael
>
>
RE: Problem with HBase and HDFS
Posted by Genady <ge...@exelate.com>.
Michael,
Did you looked at region server log at $HBASE_HOME/logs? Looks like
configuration issue.
You could look on example of single node configuration in my blog:
http://javadialog.blogspot.com/2008/12/running-hadoop-with-hbase-on-centos.h
tml
Gennady
-----Original Message-----
From: Michael Seibold [mailto:seibold@in.tum.de]
Sent: Monday, January 19, 2009 11:13 AM
To: hbase-user@hadoop.apache.org
Subject: Problem with HBase and HDFS
Hi,
I get the following error when using HBase with HDFS. When I configure
HBase to write to the local file system everything works fine.
I'm using:
hadoop-0.18.2
hbase-0.18.1
I have installed hadoop on node1:
- hadoop-site.xml:
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://node1/</value>
</property>
</configuration>
-slaves:
node1
I have installed hbase on node2:
- hbase-site.xml:
<configuration>
<property>
<name>hbase.master</name>
<value>node2:9000</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://node1:8020/hbase</value>
</property>
</configuration>
- regionservers:
node2
I format the hadoop namenode: $HADOOP_HOME/bin/hadoop namenode -format
I start hadoop on node1: $HADOOP_HOME/bin/start-dfs.sh
(I can use the browser interface to browse the HDFS, HDFS seems to work
fine)
I start hbase on node2: $HBASE_HOME/bin/start-hbase.sh
I start the hbase shell on node2: $HBASE_HOME/bin/hbase shell, enter the
command "list" in order to list the existing tables and get the
following NullPointerException:
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.18.1, r707159, Wed Oct 22 12:43:06 PDT 2008
hbase(main):001:0> list
NativeException: java.lang.NullPointerException: null
from org/apache/hadoop/hbase/client/ServerCallable.java:71:in
`getRegionName'
from
org/apache/hadoop/hbase/client/HConnectionManager.java:863:in
`getRegionServerWithRetries'
from org/apache/hadoop/hbase/client/MetaScanner.java:56:in
`metaScan'
from org/apache/hadoop/hbase/client/MetaScanner.java:30:in
`metaScan'
from
org/apache/hadoop/hbase/client/HConnectionManager.java:297:in
`listTables'
from org/apache/hadoop/hbase/client/HBaseAdmin.java:117:in
`listTables'
from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
from sun/reflect/DelegatingMethodAccessorImpl.java:25:in
`invoke'
from java/lang/reflect/Method.java:585:in `invoke'
from org/jruby/javasupport/JavaMethod.java:250:in
`invokeWithExceptionHandling'
from org/jruby/javasupport/JavaMethod.java:219:in `invoke'
from org/jruby/javasupport/JavaClass.java:416:in `execute'
from
org/jruby/internal/runtime/methods/SimpleCallbackMethod.java:67:in
`call'
from org/jruby/internal/runtime/methods/DynamicMethod.java:70:in
`call'
from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
... 131 levels...
from
ruby.home.ntap.seibold.distributed_hbase.hbase_minus_0_dot_18_dot_1.bin.hirb
Invokermethod__23$RUBY$startOpt:-1:in `call'
from org/jruby/internal/runtime/methods/DynamicMethod.java:74:in
`call'
from
org/jruby/internal/runtime/methods/CompiledMethod.java:48:in
`call'
from org/jruby/runtime/CallSite.java:123:in `cacheAndCall'
from org/jruby/runtime/CallSite.java:298:in `call'
from
ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//hom
e/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:351:in
`__file__'
from
ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//hom
e/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:-1:in
`__file__'
from
ruby/home/ntap/seibold/distributed_hbase/hbase_minus_0_dot_18_dot_1/bin//hom
e/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:-1:in
`load'
from org/jruby/Ruby.java:512:in `runScript'
from org/jruby/Ruby.java:432:in `runNormally'
from org/jruby/Ruby.java:312:in `runFromMain'
from org/jruby/Main.java:144:in `run'
from org/jruby/Main.java:89:in `run'
from org/jruby/Main.java:80:in `main'
from
/home/ntap/seibold/distributed_hbase/hbase-0.18.1/bin/../bin/hirb.rb:242:in
`list'
from (hbase):2:in `binding'hbase(main):002:0>
(In the browser interface of HDFS I can see, that the directory hbase
has been created and contains some folders and files)
What am I doing wrong?
Kind regards,
Michael