You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rakesh Kumar Rakshit <ih...@gmail.com> on 2011/04/25 11:44:58 UTC
HBase unable to find region servers
Hello friends,
I was using a hadoop cluster of apache hadoop (version 0.20.2) with apache
hbase-0.20.6(2 regionservers) and apache zookeeper 3.3.1 (cluster of 2).
I ran into problems when I replaced the apache hadoop 0.20.2 with CDH3(cloudera
hadoop). When I started HBase everything started fine i.e. HMaster,HRegionServer
etc.
But the HBase master cannot find regionservers but I can see the Regionservers
running.
Can anyone please tell me what exactly is the problem and how to solve it???
Thanks,
Rakesh Kumar Rakshit
Re: HBase unable to find region servers
Posted by Jean-Daniel Cryans <jd...@apache.org>.
>From what I can tell it's not that the master can't find the region
servers, it's the region servers that never checkin with the master.
You might want to look at their logs.
J-D
On Mon, Apr 25, 2011 at 11:03 PM, Rakesh Kumar Rakshit
<ih...@gmail.com> wrote:
> Hi,
> Here is the HMaster log :
>
> Mon Apr 25 14:51:39 IST 2011 Starting master on impetus-822
> ulimit -n 1024
BTW 1024 is bad. Check the documentation.
Re: HBase unable to find region servers
Posted by Rakesh Kumar Rakshit <ih...@gmail.com>.
Hi,
Here is the HMaster log :
Mon Apr 25 14:51:39 IST 2011 Starting master on impetus-822
ulimit -n 1024
2011-04-25 14:51:40,250 INFO org.apache.hadoop.hbase.master.HMaster:
vmName=Java
HotSpot(TM) Client VM, vmVendor=Sun Microsystems Inc., vmVersion=17.1-b03
2011-04-25 14:51:40,251 INFO org.apache.hadoop.hbase.master.HMaster:
vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError,
-XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
-XX:+HeapDumpOnOutOfMemoryError, -XX:+UseConcMarkSweepGC,
-XX:+CMSIncrementalMode, -XX:+HeapDumpOnOutOfMemoryError,
-XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode,
-Dhbase.log.dir=/opt/ladap/common/hiveHBase/hbase-0.20.6/bin/../logs,
-Dhbase.log.file=hbase-ilab-master-impetus-822.log,
-Dhbase.home.dir=/opt/ladap/common/hiveHBase/hbase-0.20.6/bin/..,
-Dhbase.id.str=ilab, -Dhbase.root.logger=INFO,DRFA]
2011-04-25 14:51:40,328 INFO org.apache.hadoop.hbase.master.HMaster:
My address is hadoop-namenode:60000
2011-04-25 14:51:40,944 DEBUG org.apache.hadoop.hbase.util.FSUtils:
Created version file at hdfs://hadoop-namenode:9000/hbase set its version at:7
2011-04-25 14:51:40,991 INFO org.apache.hadoop.hbase.master.HMaster:
BOOTSTRAP:creating ROOT and first META regions
2011-04-25 14:51:41,019 INFO org.apache.hadoop.hbase.regionserver.HLog:
HLog configuration: blocksize=67108864, rollsize=63753420, enabled=true,
flushlogentries=100, optionallogflushinternal=10000ms
2011-04-25 14:51:41,061 INFO org.apache.hadoop.hbase.regionserver.HLog:
New hlog /hbase/-ROOT-/70236052/.logs/hlog.dat.1303723301019
2011-04-25 14:51:41,062 DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Using syncFs--hadoop-4379
2011-04-25 14:51:41,075 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Creating region -ROOT-,,0, encoded=70236052
2011-04-25 14:51:41,172 INFO org.apache.hadoop.hbase.regionserver.HRegion:
region -ROOT-,,0/70236052 available; sequence id is 0
2011-04-25 14:51:41,210 INFO org.apache.hadoop.hbase.regionserver.HLog:
HLog configuration: blocksize=67108864, rollsize=63753420, enabled=true,
flushlogentries=100, optionallogflushinternal=10000ms
2011-04-25 14:51:41,219 INFO org.apache.hadoop.hbase.regionserver.HLog:
New hlog /hbase/.META./1028785192/.logs/hlog.dat.1303723301211
2011-04-25 14:51:41,219 DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Using syncFs--hadoop-4379
2011-04-25 14:51:41,220 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Creating region .META.,,1, encoded=1028785192
2011-04-25 14:51:41,265 INFO org.apache.hadoop.hbase.regionserver.HRegion:
region .META.,,1/1028785192 available; sequence id is 0
2011-04-25 14:51:41,286 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Closing -ROOT-,,0: disabling compactions & flushes
2011-04-25 14:51:41,286 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Updates disabled for region, no outstanding scanners on -ROOT-,,0
2011-04-25 14:51:41,286 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
No more row locks outstanding on region -ROOT-,,0
2011-04-25 14:51:41,312 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memstore flush for region -ROOT-,,0. Current region memstore size 576.0
2011-04-25 14:51:41,313 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished snapshotting, commencing flushing stores
2011-04-25 14:51:41,424 INFO org.apache.hadoop.hbase.regionserver.Store:
Added hdfs://hadoop-namenode:9000/hbase/-ROOT-/70236052/info/4669534531662743509,
entries=1, sequenceid=2, memsize=576.0, filesize=839.0 to -ROOT-,,0
2011-04-25 14:51:41,424 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Caches flushed, doing commit now (which includes update scanners)
2011-04-25 14:51:41,425 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~576.0 for region -ROOT-,,0 in 139ms,
sequence id=2, compaction requested=false
2011-04-25 14:51:41,425 DEBUG org.apache.hadoop.hbase.regionserver.Store:
closed info
2011-04-25 14:51:41,425 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed -ROOT-,,0
2011-04-25 14:51:41,425 DEBUG org.apache.hadoop.hbase.regionserver.HLog:
closing hlog writer in hdfs://hadoop-namenode:9000/hbase/-ROOT-/70236052/.logs
2011-04-25 14:51:41,444 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Closing .META.,,1: disabling compactions & flushes
2011-04-25 14:51:41,444 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Updates disabled for region, no outstanding scanners on .META.,,1
2011-04-25 14:51:41,444 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
No more row locks outstanding on region .META.,,1
2011-04-25 14:51:41,444 DEBUG org.apache.hadoop.hbase.regionserver.Store:
closed historian
2011-04-25 14:51:41,444 DEBUG org.apache.hadoop.hbase.regionserver.Store:
closed info
2011-04-25 14:51:41,444 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed .META.,,1
2011-04-25 14:51:41,444 DEBUG org.apache.hadoop.hbase.regionserver.HLog:
closing hlog writer in hdfs://hadoop-namenode:9000/hbase/.META./1028785192/.logs
2011-04-25 14:51:41,712 INFO org.apache.zookeeper.ClientCnxn:
zookeeper.disableAutoWatchReset is false
2011-04-25 14:51:41,723 INFO org.apache.zookeeper.ClientCnxn:
Attempting connection to server zk-server1/192.168.41.67:2181
2011-04-25 14:51:41,725 INFO org.apache.zookeeper.ClientCnxn:
Priming connection to java.nio.channels.SocketChannel[connected
local=/192.168.41.53:35933
remote=zk-server1/192.168.41.67:2181]
2011-04-25 14:51:41,740 INFO org.apache.zookeeper.ClientCnxn:
Server connection successful
2011-04-25 14:51:41,749 INFO org.apache.hadoop.hbase.master.RegionManager:
-ROOT- region unset (but not set to be reassigned)
2011-04-25 14:51:41,751 INFO org.apache.hadoop.hbase.master.RegionManager:
ROOT inserted into regionsInTransition
2011-04-25 14:51:41,768 DEBUG org.apache.hadoop.hbase.master.HMaster:
Got event None with path null
2011-04-25 14:51:41,783 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
for /hbase/master
2011-04-25 14:51:41,812 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Wrote master address
192.168.41.53:60000 to ZooKeeper
2011-04-25 14:51:41,823 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: State node exists.
2011-04-25 14:51:41,833 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper:
Read ZNode /hbase/master got 192.168.41.53:60000
2011-04-25 14:51:41,833 INFO org.apache.hadoop.hbase.master.HMaster:
HMaster initialized on 192.168.41.53:60000
2011-04-25 14:51:41,834 DEBUG org.apache.hadoop.hbase.master.HMaster:
Checking cluster state...
2011-04-25 14:51:41,835 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper:
Read ZNode /hbase/root-region-server got 192.168.41.87:60020
2011-04-25 14:51:41,842 DEBUG org.apache.hadoop.hbase.master.HMaster:
This is a fresh start, proceeding with normal startup
2011-04-25 14:51:41,845 DEBUG org.apache.hadoop.hbase.master.HMaster:
No log files to split, proceeding...
2011-04-25 14:51:41,861 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=Master, sessionId=HMaster
2011-04-25 14:51:41,876 INFO
org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized
2011-04-25 14:52:41,735 INFO org.apache.hadoop.hbase.master.ServerManager:
0 region servers, 0 dead, average load NaN
2011-04-25 14:52:41,892 INFO org.apache.hadoop.hbase.master.BaseScanner:
All 0 .META. region(s) scanned
2011-04-25 14:53:41,735 INFO org.apache.hadoop.hbase.master.ServerManager:
0 region servers, 0 dead, average load NaN
2011-04-25 14:53:41,892 INFO org.apache.hadoop.hbase.master.BaseScanner:
All 0 .META. region(s) scanned
Re: HBase unable to find region servers
Posted by Stack <st...@duboce.net>.
What exception do you see?
Please upgrade to 0.90.2 HBase.
Thanks,
St.Ack
On Mon, Apr 25, 2011 at 2:44 AM, Rakesh Kumar Rakshit
<ih...@gmail.com> wrote:
> Hello friends,
>
> I was using a hadoop cluster of apache hadoop (version 0.20.2) with apache
> hbase-0.20.6(2 regionservers) and apache zookeeper 3.3.1 (cluster of 2).
> I ran into problems when I replaced the apache hadoop 0.20.2 with CDH3(cloudera
> hadoop). When I started HBase everything started fine i.e. HMaster,HRegionServer
> etc.
> But the HBase master cannot find regionservers but I can see the Regionservers
> running.
>
> Can anyone please tell me what exactly is the problem and how to solve it???
>
> Thanks,
> Rakesh Kumar Rakshit
>
>