You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Erik Holstad <er...@gmail.com> on 2008/03/14 20:46:08 UTC
Fwd: Simple startup!
Hi I'm totally new to Hbase and Hadoop and just checking it out to see if we
can use it.
So I'm just trying to get it up and running, and after some snooping around
I think I got
Hadoop up and running, but when I try to create a table in the Hbase shell I
get an error message:
hql > create table test(col1);
08/03/14 10:44:16 WARN hbase.HConnectionManager$TableServers: Testing for
table existence threw exception
org.apache.hadoop.hbase.NoServerForRegionException: Timed out trying to
locate root region
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRootRegion(
HConnectionManager.java:718)
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegion(
HConnectionManager.java:329)
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.relocateRegion(
HConnectionManager.java:311)
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegionInMeta(
HConnectionManager.java:476)
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.locateRegion(
HConnectionManager.java:339)
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.relocateRegion(
HConnectionManager.java:311)
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.listTables(
HConnectionManager.java:291)
at
org.apache.hadoop.hbase.HConnectionManager$TableServers.tableExists(
HConnectionManager.java:227)
at org.apache.hadoop.hbase.hql.CreateCommand.execute(
CreateCommand.java:49)
at org.apache.hadoop.hbase.hql.HQLClient.executeQuery(HQLClient.java
:50)
at org.apache.hadoop.hbase.Shell.main(Shell.java:114)
Creating table... Please wait.
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String
index out of range: -1
at java.lang.String.substring(String.java:1950)
at org.apache.hadoop.hbase.hql.BasicCommand.extractErrMsg(
BasicCommand.java:64)
at org.apache.hadoop.hbase.hql.BasicCommand.extractErrMsg(
BasicCommand.java:68)
at org.apache.hadoop.hbase.hql.CreateCommand.execute(
CreateCommand.java:67)
at org.apache.hadoop.hbase.hql.HQLClient.executeQuery(HQLClient.java
:50)
at org.apache.hadoop.hbase.Shell.main(Shell.java:114)
Looking at the Hadoop and Hbase logs I found this in the
hbase-erik-regionserver-....log
2008-03-14 10:39:35,952 ERROR org.apache.hadoop.hbase.HRegionServer: Can not
start region server because java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(
NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:539)
at org.apache.hadoop.hbase.HRegionServer.doMain(HRegionServer.java
:1740)
at org.apache.hadoop.hbase.HRegionServer.main(HRegionServer.java
:1768)
Caused by: java.net.BindException: Problem binding to /127.0.0.1:60010
at org.apache.hadoop.ipc.Server.bind(Server.java:193)
at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:252)
at org.apache.hadoop.ipc.Server.<init>(Server.java:973)
at org.apache.hadoop.ipc.Server.<init>(Server.java:949)
at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.<init>(HbaseRPC.java
:397)
at org.apache.hadoop.hbase.ipc.HbaseRPC.getServer(HbaseRPC.java:367)
at org.apache.hadoop.hbase.HRegionServer.<init>(HRegionServer.java
:609)
at org.apache.hadoop.hbase.HRegionServer.<init>(HRegionServer.java
:572)
... 6 more
The other logs look like:
hadoop.namenode:
2008-03-14 10:37:47,005 INFO org.apache.hadoop.dfs.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = erik0.streamy.com/72.34.249.26
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.16.1-dev
STARTUP_MSG: build = -r ; compiled by 'erik' on Thu Mar 13 15:28:07 PDT
2008
************************************************************/
2008-03-14 10:37:47,098 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing RPC Metrics with serverName=NameNode, port=9000
2008-03-14 10:37:47,103 INFO org.apache.hadoop.dfs.NameNode: Namenode up at:
localhost/127.0.0.1:9000
2008-03-14 10:37:47,105 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=NameNode, sessionId=null
2008-03-14 10:37:47,107 INFO org.apache.hadoop.dfs.NameNodeMetrics:
Initializing NameNodeMeterics using context object:
org.apache.hadoop.metrics.spi.NullContext
2008-03-14 10:37:47,142 INFO org.apache.hadoop.fs.FSNamesystem:
fsOwner=erik,erik
2008-03-14 10:37:47,142 INFO org.apache.hadoop.fs.FSNamesystem:
supergroup=supergroup
2008-03-14 10:37:47,142 INFO org.apache.hadoop.fs.FSNamesystem:
isPermissionEnabled=true
2008-03-14 10:37:47,218 INFO org.apache.hadoop.fs.FSNamesystem: Finished
loading FSImage in 92 msecs
2008-03-14 10:37:47,219 INFO org.apache.hadoop.dfs.StateChange: STATE*
SafeModeInfo.enter: Safe mode is ON.
Safe mode will be turned off automatically.
2008-03-14 10:37:47,223 INFO org.apache.hadoop.fs.FSNamesystem: Registered
FSNamesystemStatusMBean
2008-03-14 10:37:47,346 INFO org.mortbay.util.Credential: Checking Resource
aliases
2008-03-14 10:37:47,381 INFO org.mortbay.http.HttpServer: Version
Jetty/5.1.4
2008-03-14 10:37:47,381 INFO org.mortbay.util.Container: Started
HttpContext[/static,/static]
2008-03-14 10:37:47,381 INFO org.mortbay.util.Container: Started
HttpContext[/logs,/logs]
2008-03-14 10:37:47,736 INFO org.mortbay.util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@47f08ed8
2008-03-14 10:37:47,784 INFO org.mortbay.util.Container: Started
WebApplicationContext[/,/]
2008-03-14 10:37:47,786 INFO org.mortbay.http.SocketListener: Started
SocketListener on 0.0.0.0:50070
2008-03-14 10:37:47,786 INFO org.mortbay.util.Container: Started
org.mortbay.jetty.Server@6a47a0d4
2008-03-14 10:37:47,786 INFO org.apache.hadoop.fs.FSNamesystem: Web-server
up at: 0.0.0.0:50070
2008-03-14 10:37:47,786 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2008-03-14 10:37:47,788 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 9000: starting
2008-03-14 10:37:47,788 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 9000: starting
2008-03-14 10:37:47,788 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 9000: starting
2008-03-14 10:37:47,788 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 9000: starting
2008-03-14 10:37:47,788 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 9000: starting
2008-03-14 10:37:47,788 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 9000: starting
2008-03-14 10:37:47,788 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 9000: starting
2008-03-14 10:37:47,789 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 9000: starting
2008-03-14 10:37:47,789 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 9000: starting
2008-03-14 10:37:47,789 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 9000: starting
2008-03-14 10:37:47,791 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 9000: starting
2008-03-14 10:37:51,267 INFO org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.registerDatanode: node registration from 127.0.0.1:50010 storage
DS-1091502922-72.34.249.26-50010-1205517834935
2008-03-14 10:37:51,270 INFO org.apache.hadoop.net.NetworkTopology: Adding a
new node: /default-rack/127.0.0.1:50010
2008-03-14 10:38:21,334 INFO org.apache.hadoop.fs.FSNamesystem: Leaving
safemode after 34208 msecs
2008-03-14 10:38:21,334 INFO org.apache.hadoop.dfs.StateChange: STATE*
SafeModeInfo.leave: Safe mode is OFF.
2008-03-14 10:38:21,334 INFO org.apache.hadoop.dfs.StateChange: STATE*
Network topology has 1 racks and 1 datanodes
2008-03-14 10:38:21,334 INFO org.apache.hadoop.dfs.StateChange: STATE*
UnderReplicatedBlocks has 0 blocks
2008-03-14 10:42:54,404 INFO org.apache.hadoop.fs.FSNamesystem: Roll Edit
Log from 127.0.0.1
2008-03-14 10:42:54,405 INFO org.apache.hadoop.fs.FSNamesystem: Number of
transactions: 0 Total time for transactions(ms): 0 Number of syncs: 0
SyncTimes(ms): 0
2008-03-14 10:47:54,485 INFO org.apache.hadoop.fs.FSNamesystem: Roll Edit
Log from 127.0.0.1
2008-03-14 10:52:54,533 INFO org.apache.hadoop.fs.FSNamesystem: Roll Edit
Log from 127.0.0.1
2008-03-14 10:57:54,581 INFO org.apache.hadoop.fs.FSNamesystem: Roll Edit
Log from 127.0.0.1
hadoop.datanode:
2008-03-14 10:37:50,557 INFO org.apache.hadoop.dfs.DataNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG: host = erik0.streamy.com/72.34.249.26
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.16.1-dev
STARTUP_MSG: build = -r ; compiled by 'erik' on Thu Mar 13 15:28:07 PDT
2008
************************************************************/
2008-03-14 10:37:50,736 INFO org.apache.hadoop.dfs.DataNode: Registered
FSDatasetStatusMBean
2008-03-14 10:37:50,736 INFO org.apache.hadoop.dfs.DataNode: Opened server
at 50010
2008-03-14 10:37:50,737 INFO org.apache.hadoop.dfs.DataNode: Balancing
bandwith is 1048576 bytes/s
2008-03-14 10:37:50,825 INFO org.mortbay.util.Credential: Checking Resource
aliases
2008-03-14 10:37:50,860 INFO org.mortbay.http.HttpServer: Version
Jetty/5.1.4
2008-03-14 10:37:50,861 INFO org.mortbay.util.Container: Started
HttpContext[/static,/static]
2008-03-14 10:37:50,863 INFO org.mortbay.util.Container: Started
HttpContext[/logs,/logs]
2008-03-14 10:37:51,175 INFO org.mortbay.util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@a0b1cd0
2008-03-14 10:37:51,263 INFO org.mortbay.util.Container: Started
WebApplicationContext[/,/]
2008-03-14 10:37:51,265 INFO org.mortbay.http.SocketListener: Started
SocketListener on 0.0.0.0:50075
2008-03-14 10:37:51,265 INFO org.mortbay.util.Container: Started
org.mortbay.jetty.Server@7aae3364
2008-03-14 10:37:51,274 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=DataNode, sessionId=null
2008-03-14 10:37:51,282 INFO org.apache.hadoop.dfs.DataNode: 127.0.0.1:50010In
DataNode.run, data = FSDataset{dirpath='/tmp/hadoop-erik/dfs/data/current'}
2008-03-14 10:37:51,286 INFO org.apache.hadoop.dfs.DataNode: using
BLOCKREPORT_INTERVAL of 3266197msec Initial delay: 60000msec
2008-03-14 10:37:51,332 INFO org.apache.hadoop.dfs.DataNode: BlockReport of
5 blocks got processed in 28 msecs
2008-03-14 10:39:32,994 INFO org.apache.hadoop.dfs.DataNode:
127.0.0.1:50010Served block blk_5906536682811187945 to /127.0.0.1
hbase.master:
2008-03-14 10:39:32,926 INFO org.apache.hadoop.hbase.HMaster: Root region
dir: /tmp/hadoop-erik/hbase/-ROOT-/70236052
2008-03-14 10:39:33,054 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing RPC Metrics with serverName=60000, port=60000
2008-03-14 10:39:33,143 INFO org.apache.hadoop.hbase.HMaster: HMaster
initialized on 127.0.0.1:60000
2008-03-14 10:39:33,215 INFO org.mortbay.util.Credential: Checking Resource
aliases
2008-03-14 10:39:33,266 INFO org.mortbay.http.HttpServer: Version
Jetty/5.1.4
2008-03-14 10:39:33,266 INFO org.mortbay.util.Container: Started
HttpContext[/static,/static]
2008-03-14 10:39:33,266 INFO org.mortbay.util.Container: Started
HttpContext[/logs,/logs]
2008-03-14 10:39:33,548 INFO org.mortbay.util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@6b9cd75a
2008-03-14 10:39:33,656 INFO org.mortbay.util.Container: Started
WebApplicationContext[/,/]
2008-03-14 10:39:33,860 INFO org.mortbay.util.Container: Started
org.mortbay.jetty.servlet.WebApplicationHandler@e0d5eb7
2008-03-14 10:39:33,863 INFO org.mortbay.util.Container: Started
WebApplicationContext[/api,rest]
2008-03-14 10:39:33,865 INFO org.mortbay.http.SocketListener: Started
SocketListener on 0.0.0.0:60010
2008-03-14 10:39:33,865 INFO org.mortbay.util.Container: Started
org.mortbay.jetty.Server@877ef83
2008-03-14 10:39:33,865 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2008-03-14 10:39:33,867 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 60000: starting
2008-03-14 10:39:33,867 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 60000: starting
2008-03-14 10:39:33,868 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 60000: starting
2008-03-14 10:39:33,869 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 60000: starting
And my .site files look like:
hadoop:
<property>
<name>fs.default.name</name>
<value>localhost:9000</value>
<description>Namenode </description>
</property>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
<description>JobTracker </description>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
hbase:
<property>
<name>hbase.master</name>
<value>localhost:60000</value>
<description>The host and port that the HBase master runs at.
TODO: Support 'local' (All running in single context).
</description>
</property>
<property>
<name>hbase.regionserver</name>
<value>localhost:60010</value>
<description>The host and port a HBase region server runs at.
</description>
</property>
Regards Erik
Re: Simple startup!
Posted by Erik Holstad <er...@gmail.com>.
I had the : there from the beginning but I tried again, and also made a new
table to see if that would help, but not.
When trying to insert a value into a column and row the following line is
used:
curl -v -T /tmp/diff.txt http://localhost:60010/api/y/row/1?column=y:1
where diff.txt is just the > from the output of:
curl -v http://localhost:60010/api/y/row/1 > /tmp/diff.txt
don't think that this could be right, but not sure where to get and put the
files.
Regards Erik
Re: Simple startup!
Posted by stack <st...@duboce.net>.
The doc. on the wiki may be stale. Can you add the ':' to the column
family name and see if it works then?
St.Ack
Erik Holstad wrote:
> Hey!
> Got everything to work , it was just a silly DNS problem, but it is now
> fixed.
> But now when I'm trying to access Hbase via the rest interface I get:
>
> <title>Error 500 org.apache.hadoop.hbase.InvalidColumnNameException:
> eDo=
> is missing the colon family/qualifier separator
>
> and I'm just trying to follow the instructions on:
> http://wiki.apache.org/hadoop/Hbase/HbaseRest
> and https://issues.apache.org/jira/browse/HBASE-269
>
> Is there another simple way to get started with it?
>
> Regards Erik
>
>
Re: Simple startup!
Posted by Erik Holstad <er...@gmail.com>.
Hey!
Got everything to work , it was just a silly DNS problem, but it is now
fixed.
But now when I'm trying to access Hbase via the rest interface I get:
<title>Error 500 org.apache.hadoop.hbase.InvalidColumnNameException:
eDo=
is missing the colon family/qualifier separator
and I'm just trying to follow the instructions on:
http://wiki.apache.org/hadoop/Hbase/HbaseRest
and https://issues.apache.org/jira/browse/HBASE-269
Is there another simple way to get started with it?
Regards Erik
Re: Simple startup!
Posted by Erik Holstad <er...@gmail.com>.
Hey!
I got hbase running on both the new Kubuntu and the old Ubuntu, but not on
the new Ubuntu and still the same problem
with Fedora 8. I managed to start hbase without hadoop last week, but today
I'm back to have to start hadoop, so I guess
there is something that I'm missing now.
Erik
Re: Simple startup!
Posted by Erik Holstad <er...@gmail.com>.
Hi!
Not really, but I will try on Hardy today, I think that the issue is like
said earlier, with some kind of firewall.
But I didn't get it to work without Hadoop either, it just said the same
thing while building the table.
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://192.168.249.65:9000/hbase</value>
<description></description>
</property>
<property>
<name>hbase.master</name>
<value>192.168.249.65:60000</value>
<description>The host and port that the HBase master runs
at.
TODO: Support 'local' (All running in single context).
</description>
</property>
</configuration>
with this site file
Regards Erik
Re: Simple startup!
Posted by stack <st...@duboce.net>.
Any luck configuring your network Erik (after our IRC chat)?
St.Ack
Erik Holstad wrote:
> So this is what I did!
> Installed Hadoop according to the instructions on the
> http://www.jakobhoman.com/2007/10/how-to-configure-single-node-hadoop.htmlsite
> After that I started a hadoop client with start-all.sh,
> Looked at the log files for Hadoop, and the Interface in Firefox, and
> everything looked good.
> But when I started the Hbase and looked at that log file, it couldn't
> connect.
>
> So then I changed the localhost argument in the hadoop-site.xml to my ip
> 192....
> and the same thing in the hbase-site file
>
> Regards Erik
>
>
Re: Simple startup!
Posted by Erik Holstad <er...@gmail.com>.
So this is what I did!
Installed Hadoop according to the instructions on the
http://www.jakobhoman.com/2007/10/how-to-configure-single-node-hadoop.htmlsite
After that I started a hadoop client with start-all.sh,
Looked at the log files for Hadoop, and the Interface in Firefox, and
everything looked good.
But when I started the Hbase and looked at that log file, it couldn't
connect.
So then I changed the localhost argument in the hadoop-site.xml to my ip
192....
and the same thing in the hbase-site file
Regards Erik
Re: Simple startup!
Posted by stack <st...@duboce.net>.
Tell us more Erik.
You changed hbase.master to be some IP?
Then you started hbase and an hbase client was unable to connect to the
hbase master? Was client on same machine as the master? Did the client
hbase-site.xml have the master's IP?
St.Ack
Erik Holstad wrote:
> Hi!
> I tried to change the localhost argument to the actual address found in the
> /etc/hosts file
> but I ended up with the same thing as before, that Hbase was unable to
> connect to
> the database. The weird thing is that I installed it the same way on an
> Ubuntu machine
> and there it worked perfectly.
> Regards Erik
>
>
Re: Simple startup!
Posted by Erik Holstad <er...@gmail.com>.
Hi!
I tried to change the localhost argument to the actual address found in the
/etc/hosts file
but I ended up with the same thing as before, that Hbase was unable to
connect to
the database. The weird thing is that I installed it the same way on an
Ubuntu machine
and there it worked perfectly.
Regards Erik
Re: Simple startup!
Posted by Erwan Arzur <ea...@gmail.com>.
How does it work if you change localhost to erik0.streamy.com (provided you
have properly setup DNS) in hadoop-site.xml ?
It seems the hadoop lib makes lots or use of InetAddress.getLocalHost() to
setup default listening sockets, which doesn't return 127.0.0.1. So having
the configuration files using localhost just confuses the daemons.
Make sure you have these setup too:
HBASE_CONF_DIR=<hbase config dir>
HADOOP_CONF_DIR=<hadoop config dir>
It seems like your hbase shell doesn't know where to connect to.