You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by jayavelu jaisenthilkumar <jo...@gmail.com> on 2010/03/31 18:24:30 UTC
Failed to create /hbase.... KeeperErrorCode = ConnectionLoss for
/hbase
Hi,
I am using 1 master and 2 slaves one has password for ssh.
I am using hadoop0.20.1 and hbase0.20.3(direct one not upgraded one)
1)The slave one with password is could not be disabled, i removed the whole
.ssh directory try to ssh-keygen with passwordless phrase, still i am asked
for the password when i
"ssh localhost"
2) I am able to run hadoop and successfuly run the Mapreduce in the hadoop
environment as per the Running Hadoop On Ubuntu Linux (Multi-Node Cluster)
by noel
3) I am now following the tutorial hbase: overview HBase 0.20.3 API
Its not clearly stated as the mulitnode cluster hadoop for the distributed
mode hbase.
I ran the hdfs and the hbase using start-dfs.sh and start-hbase.sh
respectively.
The master log indicates connection loss on the /hbase : ( is this hbase is
created by Hbase or should we do to create it again
2010-03-31 16:45:57,850 INFO org.apache.zookeeper.
ClientCnxn: Attempting connection to server Hadoopserver/192.168.1.65:2222
2010-03-31 16:45:57,858 INFO org.apache.zookeeper.ClientCnxn: Priming
connection to java.nio.channels.SocketChannel[connected local=/
192.168.1.65:43017 remote=Hadoopserver/192.168.1.65:2222]
2010-03-31 16:45:57,881 INFO org.apache.zookeeper.ClientCnxn: Server
connection successful
2010-03-31 16:45:57,883 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@11c2b67
java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
lim=4 cap=4]
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown input
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at
sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown output
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at
sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-03-31 16:45:57,933 INFO org.apache.hadoop.hbase.master.RegionManager:
-ROOT- region unset (but not set to be reassigned)
2010-03-31 16:45:57,934 INFO org.apache.hadoop.hbase.master.RegionManager:
ROOT inserted into regionsInTransition
2010-03-31 16:45:58,024 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase/master
2010-03-31 16:45:58,422 INFO org.apache.zookeeper.ClientCnxn: Attempting
connection to server Hadoopclient1/192.168.1.2:2222
2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Priming
connection to java.nio.channels.SocketChannel[connected local=/
192.168.1.65:51219 remote=Hadoopclient1/192.168.1.2:2222]
2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Server
connection successful
2010-03-31 16:45:58,436 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@17b6643
java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
lim=4 cap=4]
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown input
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at
sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown output
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at
sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-03-31 16:45:58,537 WARN
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to create /hbase
-- check quorum servers,
currently=Hadoopclient1:2222,Hadoopclient:2222,Hadoopserver:2222
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:608)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureExists(ZooKeeperWrapper.java:405)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureParentExists(ZooKeeperWrapper.java:428)
at
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.writeMasterAddress(ZooKeeperWrapper.java:516)
at
org.apache.hadoop.hbase.master.HMaster.writeAddressToZooKeeper(HMaster.java:263)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:245)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1241)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1282)
2010-03-31 16:45:58,549 INFO org.apache.zookeeper.ClientCnxn: Attempting
connection to server Hadoopclient/192.168.1.3:2222
2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Priming
connection to java.nio.channels.SocketChannel[connected local=/
192.168.1.65:56142 remote=Hadoopclient/192.168.1.3:2222]
2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Server
connection successful
2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@76e8a7
java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
lim=4 cap=4]
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown input
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at
sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown output
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at
sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-03-31 16:45:58,678 DEBUG
org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase/master
my hbase-site.xml ( Master)
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://Hadoopserver:54310/hbase</value>
<description>The directory shared by region servers.
Should be fully-qualified to include the filesystem to use.
E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
</description>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
<description>The mode the cluster will be in. Possible values are
false: standalone and pseudo-distributed setups with managed Zookeeper
true: fully-distributed with unmanaged Zookeeper Quorum (see
hbase-env.sh)
</description>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
<description>Comma separated list of servers in the ZooKeeper Quorum.
For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
By default this is set to localhost for local and pseudo-distributed
modes
of operation. For a fully-distributed setup, this should be set to a
full
list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
hbase-env.sh
this is the list of servers which we will start/stop ZooKeeper on.
</description>
</property>
<property>
<name>hbase.zookeeper.property.tickTime</name>
<value>1</value>
<description>Property from ZooKeeper's config zoo.cfg.
The number of milliseconds of each tick. See
zookeeper.session.timeout description.
</description>
</property>
<property>
<name>zookeeper.retries</name>
<value>5</value>
<description>How many times to retry connections to ZooKeeper. Used for
reading/writing root region location, checking/writing out of safe
mode.
Used together with ${zookeeper.pause} in an exponential backoff
fashion
when making queries to ZooKeeper.
</description>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2222</value>
<description>Property from ZooKeeper's config zoo.cfg.
the port at which the clients will connect.
</description>
</property>
<property>
<name>dfs.replication</name>
<value>3</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
</configuration>
The hbase-site.xml in slave1, slave2
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://Hadoopserver:54310/hbase</value>
<description>The directory shared by region servers.
Should be fully-qualified to include the filesystem to use.
E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
</description>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
<description>The mode the cluster will be in. Possible values are
false: standalone and pseudo-distributed setups with managed Zookeeper
true: fully-distributed with unmanaged Zookeeper Quorum (see
hbase-env.sh)
</description>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
<description>Comma separated list of servers in the ZooKeeper Quorum.
For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
By default this is set to localhost for local and pseudo-distributed
modes
of operation. For a fully-distributed setup, this should be set to a
full
list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
hbase-env.sh
this is the list of servers which we will start/stop ZooKeeper on.
</description>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2222</value>
<description>Property from ZooKeeper's config zoo.cfg.
the port at which the clients will connect.
</description>
</property>
<property>
<name>dfs.replication</name>
<value>3</value>
<description>Default block replication.
The actual number of replications can be specified when the file is created.
The default is used if replication is not specified in create time.
</description>
</property>
</configuration>
Regionservers ( only on Master in my case Hadoopserver)
Hadoopserver
Hadoopclient1
Hadoopclient
Regionservers ( on slaves)
localhost
Blocked with this error for the past one week. Googled so much didnt get any
solution.
REgs.
senthil
Re: Failed to create /hbase.... KeeperErrorCode = ConnectionLoss
for /hbase
Posted by Patrick Hunt <ph...@apache.org>.
JD beat me to it. He's much more an expert on this type of issue in
hbase so take a look at his suggestions first, if there's still an issue
consider my response.
Patrick
Patrick Hunt wrote:
> If you look at the ZooKeeper server logs for the same time period do you
> see anything?
>
> The client makes a connection to the server (tcp) but cannot establish a
> session. This could be because you don't have a functioning quorum, or
> perhaps maxClientCnxns is exceeded (does the log mention "Too many
> connections from" anywhere?), etc... the server logs will shed more light.
>
> Patrick
>
> jayavelu jaisenthilkumar wrote:
>> Hi,
>> I am using 1 master and 2 slaves one has password for ssh.
>>
>> I am using hadoop0.20.1 and hbase0.20.3(direct one not upgraded one)
>>
>> 1)The slave one with password is could not be disabled, i removed the
>> whole
>> .ssh directory try to ssh-keygen with passwordless phrase, still i am
>> asked
>> for the password when i
>> "ssh localhost"
>>
>> 2) I am able to run hadoop and successfuly run the Mapreduce in the
>> hadoop
>> environment as per the Running Hadoop On Ubuntu Linux (Multi-Node
>> Cluster)
>> by noel
>>
>> 3) I am now following the tutorial hbase: overview HBase 0.20.3 API
>>
>> Its not clearly stated as the mulitnode cluster hadoop for the
>> distributed
>> mode hbase.
>>
>> I ran the hdfs and the hbase using start-dfs.sh and start-hbase.sh
>> respectively.
>>
>> The master log indicates connection loss on the /hbase : ( is this
>> hbase is
>> created by Hbase or should we do to create it again
>>
>> 2010-03-31 16:45:57,850 INFO org.apache.zookeeper.
>> ClientCnxn: Attempting connection to server
>> Hadoopserver/192.168.1.65:2222
>> 2010-03-31 16:45:57,858 INFO org.apache.zookeeper.ClientCnxn: Priming
>> connection to java.nio.channels.SocketChannel[connected local=/
>> 192.168.1.65:43017 remote=Hadoopserver/192.168.1.65:2222]
>> 2010-03-31 16:45:57,881 INFO org.apache.zookeeper.ClientCnxn: Server
>> connection successful
>> 2010-03-31 16:45:57,883 WARN org.apache.zookeeper.ClientCnxn: Exception
>> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@11c2b67
>> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
>> lim=4 cap=4]
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
>> 2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
>> exception during shutdown input
>> java.net.SocketException: Transport endpoint is not connected
>> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
>> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>> 2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
>> exception during shutdown output
>> java.net.SocketException: Transport endpoint is not connected
>> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
>> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>> 2010-03-31 16:45:57,933 INFO
>> org.apache.hadoop.hbase.master.RegionManager:
>> -ROOT- region unset (but not set to be reassigned)
>> 2010-03-31 16:45:57,934 INFO
>> org.apache.hadoop.hbase.master.RegionManager:
>> ROOT inserted into regionsInTransition
>> 2010-03-31 16:45:58,024 DEBUG
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss for /hbase/master
>> 2010-03-31 16:45:58,422 INFO org.apache.zookeeper.ClientCnxn: Attempting
>> connection to server Hadoopclient1/192.168.1.2:2222
>> 2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Priming
>> connection to java.nio.channels.SocketChannel[connected local=/
>> 192.168.1.65:51219 remote=Hadoopclient1/192.168.1.2:2222]
>> 2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Server
>> connection successful
>> 2010-03-31 16:45:58,436 WARN org.apache.zookeeper.ClientCnxn: Exception
>> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@17b6643
>> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
>> lim=4 cap=4]
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
>> 2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
>> exception during shutdown input
>> java.net.SocketException: Transport endpoint is not connected
>> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
>> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>> 2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
>> exception during shutdown output
>> java.net.SocketException: Transport endpoint is not connected
>> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
>> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>> 2010-03-31 16:45:58,537 WARN
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to create
>> /hbase
>> -- check quorum servers,
>> currently=Hadoopclient1:2222,Hadoopclient:2222,Hadoopserver:2222
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss for /hbase
>> at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
>> at
>> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:608)
>> at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureExists(ZooKeeperWrapper.java:405)
>>
>> at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureParentExists(ZooKeeperWrapper.java:428)
>>
>> at
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.writeMasterAddress(ZooKeeperWrapper.java:516)
>>
>> at
>> org.apache.hadoop.hbase.master.HMaster.writeAddressToZooKeeper(HMaster.java:263)
>>
>> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:245)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>> at
>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>>
>> at
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>>
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1241)
>> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1282)
>> 2010-03-31 16:45:58,549 INFO org.apache.zookeeper.ClientCnxn: Attempting
>> connection to server Hadoopclient/192.168.1.3:2222
>> 2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Priming
>> connection to java.nio.channels.SocketChannel[connected local=/
>> 192.168.1.65:56142 remote=Hadoopclient/192.168.1.3:2222]
>> 2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Server
>> connection successful
>> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Exception
>> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@76e8a7
>> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
>> lim=4 cap=4]
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
>> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
>> exception during shutdown input
>> java.net.SocketException: Transport endpoint is not connected
>> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
>> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
>> exception during shutdown output
>> java.net.SocketException: Transport endpoint is not connected
>> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
>> at
>> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
>> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
>> at
>> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
>> 2010-03-31 16:45:58,678 DEBUG
>> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
>> org.apache.zookeeper.KeeperException$ConnectionLossException:
>> KeeperErrorCode = ConnectionLoss for /hbase/master
>>
>> my hbase-site.xml ( Master)
>> <configuration>
>> <property>
>> <name>hbase.rootdir</name>
>> <value>hdfs://Hadoopserver:54310/hbase</value>
>> <description>The directory shared by region servers.
>> Should be fully-qualified to include the filesystem to use.
>> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
>> </description>
>> </property>
>> <property>
>> <name>hbase.cluster.distributed</name>
>> <value>true</value>
>> <description>The mode the cluster will be in. Possible values are
>> false: standalone and pseudo-distributed setups with managed
>> Zookeeper
>> true: fully-distributed with unmanaged Zookeeper Quorum (see
>> hbase-env.sh)
>> </description>
>> </property>
>> <property>
>> <name>hbase.zookeeper.quorum</name>
>> <value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
>> <description>Comma separated list of servers in the ZooKeeper Quorum.
>> For example,
>> "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
>> By default this is set to localhost for local and pseudo-distributed
>> modes
>> of operation. For a fully-distributed setup, this should be set to a
>> full
>> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
>> hbase-env.sh
>> this is the list of servers which we will start/stop ZooKeeper on.
>> </description>
>> </property>
>> <property>
>> <name>hbase.zookeeper.property.tickTime</name>
>> <value>1</value>
>> <description>Property from ZooKeeper's config zoo.cfg.
>> The number of milliseconds of each tick. See
>> zookeeper.session.timeout description.
>> </description>
>> </property>
>> <property>
>> <name>zookeeper.retries</name>
>> <value>5</value>
>> <description>How many times to retry connections to ZooKeeper.
>> Used for
>> reading/writing root region location, checking/writing out of safe
>> mode.
>> Used together with ${zookeeper.pause} in an exponential backoff
>> fashion
>> when making queries to ZooKeeper.
>> </description>
>> </property>
>> <property>
>> <name>hbase.zookeeper.property.clientPort</name>
>> <value>2222</value>
>> <description>Property from ZooKeeper's config zoo.cfg.
>> the port at which the clients will connect.
>> </description>
>> </property>
>>
>> <property>
>> <name>dfs.replication</name>
>> <value>3</value>
>> <description>Default block replication.
>> The actual number of replications can be specified when the file is
>> created.
>> The default is used if replication is not specified in create time.
>> </description>
>> </property>
>> </configuration>
>>
>>
>> The hbase-site.xml in slave1, slave2
>> <configuration>
>> <property>
>> <name>hbase.rootdir</name>
>> <value>hdfs://Hadoopserver:54310/hbase</value>
>> <description>The directory shared by region servers.
>> Should be fully-qualified to include the filesystem to use.
>> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
>> </description>
>> </property>
>> <property>
>> <name>hbase.cluster.distributed</name>
>> <value>true</value>
>> <description>The mode the cluster will be in. Possible values are
>> false: standalone and pseudo-distributed setups with managed
>> Zookeeper
>> true: fully-distributed with unmanaged Zookeeper Quorum (see
>> hbase-env.sh)
>> </description>
>> </property>
>> <property>
>> <name>hbase.zookeeper.quorum</name>
>> <value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
>> <description>Comma separated list of servers in the ZooKeeper Quorum.
>> For example,
>> "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
>> By default this is set to localhost for local and pseudo-distributed
>> modes
>> of operation. For a fully-distributed setup, this should be set to a
>> full
>> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
>> hbase-env.sh
>> this is the list of servers which we will start/stop ZooKeeper on.
>> </description>
>> </property>
>>
>> <property>
>> <name>hbase.zookeeper.property.clientPort</name>
>> <value>2222</value>
>> <description>Property from ZooKeeper's config zoo.cfg.
>> the port at which the clients will connect.
>> </description>
>> </property>
>>
>> <property>
>> <name>dfs.replication</name>
>> <value>3</value>
>> <description>Default block replication.
>> The actual number of replications can be specified when the file is
>> created.
>> The default is used if replication is not specified in create time.
>> </description>
>> </property>
>> </configuration>
>>
>> Regionservers ( only on Master in my case Hadoopserver)
>>
>> Hadoopserver
>> Hadoopclient1
>> Hadoopclient
>>
>>
>> Regionservers ( on slaves)
>> localhost
>>
>> Blocked with this error for the past one week. Googled so much didnt
>> get any
>> solution.
>>
>>
>> REgs.
>> senthil
>>
Re: Failed to create /hbase.... KeeperErrorCode = ConnectionLoss
for /hbase
Posted by Patrick Hunt <ph...@apache.org>.
If you look at the ZooKeeper server logs for the same time period do you
see anything?
The client makes a connection to the server (tcp) but cannot establish a
session. This could be because you don't have a functioning quorum, or
perhaps maxClientCnxns is exceeded (does the log mention "Too many
connections from" anywhere?), etc... the server logs will shed more light.
Patrick
jayavelu jaisenthilkumar wrote:
> Hi,
> I am using 1 master and 2 slaves one has password for ssh.
>
> I am using hadoop0.20.1 and hbase0.20.3(direct one not upgraded one)
>
> 1)The slave one with password is could not be disabled, i removed the whole
> .ssh directory try to ssh-keygen with passwordless phrase, still i am asked
> for the password when i
> "ssh localhost"
>
> 2) I am able to run hadoop and successfuly run the Mapreduce in the hadoop
> environment as per the Running Hadoop On Ubuntu Linux (Multi-Node Cluster)
> by noel
>
> 3) I am now following the tutorial hbase: overview HBase 0.20.3 API
>
> Its not clearly stated as the mulitnode cluster hadoop for the distributed
> mode hbase.
>
> I ran the hdfs and the hbase using start-dfs.sh and start-hbase.sh
> respectively.
>
> The master log indicates connection loss on the /hbase : ( is this hbase is
> created by Hbase or should we do to create it again
>
> 2010-03-31 16:45:57,850 INFO org.apache.zookeeper.
> ClientCnxn: Attempting connection to server Hadoopserver/192.168.1.65:2222
> 2010-03-31 16:45:57,858 INFO org.apache.zookeeper.ClientCnxn: Priming
> connection to java.nio.channels.SocketChannel[connected local=/
> 192.168.1.65:43017 remote=Hadoopserver/192.168.1.65:2222]
> 2010-03-31 16:45:57,881 INFO org.apache.zookeeper.ClientCnxn: Server
> connection successful
> 2010-03-31 16:45:57,883 WARN org.apache.zookeeper.ClientCnxn: Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@11c2b67
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
> lim=4 cap=4]
> at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
> 2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown output
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:57,933 INFO org.apache.hadoop.hbase.master.RegionManager:
> -ROOT- region unset (but not set to be reassigned)
> 2010-03-31 16:45:57,934 INFO org.apache.hadoop.hbase.master.RegionManager:
> ROOT inserted into regionsInTransition
> 2010-03-31 16:45:58,024 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2010-03-31 16:45:58,422 INFO org.apache.zookeeper.ClientCnxn: Attempting
> connection to server Hadoopclient1/192.168.1.2:2222
> 2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Priming
> connection to java.nio.channels.SocketChannel[connected local=/
> 192.168.1.65:51219 remote=Hadoopclient1/192.168.1.2:2222]
> 2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Server
> connection successful
> 2010-03-31 16:45:58,436 WARN org.apache.zookeeper.ClientCnxn: Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@17b6643
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
> lim=4 cap=4]
> at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
> 2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown output
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,537 WARN
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to create /hbase
> -- check quorum servers,
> currently=Hadoopclient1:2222,Hadoopclient:2222,Hadoopserver:2222
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:608)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureExists(ZooKeeperWrapper.java:405)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureParentExists(ZooKeeperWrapper.java:428)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.writeMasterAddress(ZooKeeperWrapper.java:516)
> at
> org.apache.hadoop.hbase.master.HMaster.writeAddressToZooKeeper(HMaster.java:263)
> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:245)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1241)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1282)
> 2010-03-31 16:45:58,549 INFO org.apache.zookeeper.ClientCnxn: Attempting
> connection to server Hadoopclient/192.168.1.3:2222
> 2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Priming
> connection to java.nio.channels.SocketChannel[connected local=/
> 192.168.1.65:56142 remote=Hadoopclient/192.168.1.3:2222]
> 2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Server
> connection successful
> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@76e8a7
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
> lim=4 cap=4]
> at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown output
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,678 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
>
> my hbase-site.xml ( Master)
> <configuration>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://Hadoopserver:54310/hbase</value>
> <description>The directory shared by region servers.
> Should be fully-qualified to include the filesystem to use.
> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> </description>
> </property>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> <description>The mode the cluster will be in. Possible values are
> false: standalone and pseudo-distributed setups with managed Zookeeper
> true: fully-distributed with unmanaged Zookeeper Quorum (see
> hbase-env.sh)
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
> <description>Comma separated list of servers in the ZooKeeper Quorum.
> For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
> By default this is set to localhost for local and pseudo-distributed
> modes
> of operation. For a fully-distributed setup, this should be set to a
> full
> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> hbase-env.sh
> this is the list of servers which we will start/stop ZooKeeper on.
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.property.tickTime</name>
> <value>1</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> The number of milliseconds of each tick. See
> zookeeper.session.timeout description.
> </description>
> </property>
> <property>
> <name>zookeeper.retries</name>
> <value>5</value>
> <description>How many times to retry connections to ZooKeeper. Used for
> reading/writing root region location, checking/writing out of safe
> mode.
> Used together with ${zookeeper.pause} in an exponential backoff
> fashion
> when making queries to ZooKeeper.
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.property.clientPort</name>
> <value>2222</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> the port at which the clients will connect.
> </description>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>3</value>
> <description>Default block replication.
> The actual number of replications can be specified when the file is created.
> The default is used if replication is not specified in create time.
> </description>
> </property>
> </configuration>
>
>
> The hbase-site.xml in slave1, slave2
> <configuration>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://Hadoopserver:54310/hbase</value>
> <description>The directory shared by region servers.
> Should be fully-qualified to include the filesystem to use.
> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> </description>
> </property>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> <description>The mode the cluster will be in. Possible values are
> false: standalone and pseudo-distributed setups with managed Zookeeper
> true: fully-distributed with unmanaged Zookeeper Quorum (see
> hbase-env.sh)
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
> <description>Comma separated list of servers in the ZooKeeper Quorum.
> For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
> By default this is set to localhost for local and pseudo-distributed
> modes
> of operation. For a fully-distributed setup, this should be set to a
> full
> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> hbase-env.sh
> this is the list of servers which we will start/stop ZooKeeper on.
> </description>
> </property>
>
> <property>
> <name>hbase.zookeeper.property.clientPort</name>
> <value>2222</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> the port at which the clients will connect.
> </description>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>3</value>
> <description>Default block replication.
> The actual number of replications can be specified when the file is created.
> The default is used if replication is not specified in create time.
> </description>
> </property>
> </configuration>
>
> Regionservers ( only on Master in my case Hadoopserver)
>
> Hadoopserver
> Hadoopclient1
> Hadoopclient
>
>
> Regionservers ( on slaves)
> localhost
>
> Blocked with this error for the past one week. Googled so much didnt get any
> solution.
>
>
> REgs.
> senthil
>
Re: Failed to create /hbase.... KeeperErrorCode = ConnectionLoss for
/hbase
Posted by Jean-Daniel Cryans <jd...@apache.org>.
You set the tick time like this:
<property>
<name>hbase.zookeeper.property.tickTime</name>
<value>1</value>
<description>Property from ZooKeeper's config zoo.cfg.
The number of milliseconds of each tick. See
zookeeper.session.timeout description.
</description>
1 means HBase has to report to zookeeper every 1 millisecond and if
for any reason it doesn't after 20ms, the session is expired (!!). I
recommend using the default value.
Also you should keep the same config on every node, rsync can do wonders.
J-D
On Wed, Mar 31, 2010 at 9:24 AM, jayavelu jaisenthilkumar
<jo...@gmail.com> wrote:
> Hi,
> I am using 1 master and 2 slaves one has password for ssh.
>
> I am using hadoop0.20.1 and hbase0.20.3(direct one not upgraded one)
>
> 1)The slave one with password is could not be disabled, i removed the whole
> .ssh directory try to ssh-keygen with passwordless phrase, still i am asked
> for the password when i
> "ssh localhost"
>
> 2) I am able to run hadoop and successfuly run the Mapreduce in the hadoop
> environment as per the Running Hadoop On Ubuntu Linux (Multi-Node Cluster)
> by noel
>
> 3) I am now following the tutorial hbase: overview HBase 0.20.3 API
>
> Its not clearly stated as the mulitnode cluster hadoop for the distributed
> mode hbase.
>
> I ran the hdfs and the hbase using start-dfs.sh and start-hbase.sh
> respectively.
>
> The master log indicates connection loss on the /hbase : ( is this hbase is
> created by Hbase or should we do to create it again
>
> 2010-03-31 16:45:57,850 INFO org.apache.zookeeper.
> ClientCnxn: Attempting connection to server Hadoopserver/192.168.1.65:2222
> 2010-03-31 16:45:57,858 INFO org.apache.zookeeper.ClientCnxn: Priming
> connection to java.nio.channels.SocketChannel[connected local=/
> 192.168.1.65:43017 remote=Hadoopserver/192.168.1.65:2222]
> 2010-03-31 16:45:57,881 INFO org.apache.zookeeper.ClientCnxn: Server
> connection successful
> 2010-03-31 16:45:57,883 WARN org.apache.zookeeper.ClientCnxn: Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@11c2b67
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
> lim=4 cap=4]
> at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
> 2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:57,885 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown output
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:57,933 INFO org.apache.hadoop.hbase.master.RegionManager:
> -ROOT- region unset (but not set to be reassigned)
> 2010-03-31 16:45:57,934 INFO org.apache.hadoop.hbase.master.RegionManager:
> ROOT inserted into regionsInTransition
> 2010-03-31 16:45:58,024 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
> 2010-03-31 16:45:58,422 INFO org.apache.zookeeper.ClientCnxn: Attempting
> connection to server Hadoopclient1/192.168.1.2:2222
> 2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Priming
> connection to java.nio.channels.SocketChannel[connected local=/
> 192.168.1.65:51219 remote=Hadoopclient1/192.168.1.2:2222]
> 2010-03-31 16:45:58,423 INFO org.apache.zookeeper.ClientCnxn: Server
> connection successful
> 2010-03-31 16:45:58,436 WARN org.apache.zookeeper.ClientCnxn: Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@17b6643
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
> lim=4 cap=4]
> at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
> 2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,437 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown output
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,537 WARN
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to create /hbase
> -- check quorum servers,
> currently=Hadoopclient1:2222,Hadoopclient:2222,Hadoopserver:2222
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:608)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureExists(ZooKeeperWrapper.java:405)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.ensureParentExists(ZooKeeperWrapper.java:428)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.writeMasterAddress(ZooKeeperWrapper.java:516)
> at
> org.apache.hadoop.hbase.master.HMaster.writeAddressToZooKeeper(HMaster.java:263)
> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:245)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1241)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1282)
> 2010-03-31 16:45:58,549 INFO org.apache.zookeeper.ClientCnxn: Attempting
> connection to server Hadoopclient/192.168.1.3:2222
> 2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Priming
> connection to java.nio.channels.SocketChannel[connected local=/
> 192.168.1.65:56142 remote=Hadoopclient/192.168.1.3:2222]
> 2010-03-31 16:45:58,550 INFO org.apache.zookeeper.ClientCnxn: Server
> connection successful
> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@76e8a7
> java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0
> lim=4 cap=4]
> at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:640)
> at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,577 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown output
> java.net.SocketException: Transport endpoint is not connected
> at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:651)
> at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:368)
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
> 2010-03-31 16:45:58,678 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Failed to read:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
>
> my hbase-site.xml ( Master)
> <configuration>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://Hadoopserver:54310/hbase</value>
> <description>The directory shared by region servers.
> Should be fully-qualified to include the filesystem to use.
> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> </description>
> </property>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> <description>The mode the cluster will be in. Possible values are
> false: standalone and pseudo-distributed setups with managed Zookeeper
> true: fully-distributed with unmanaged Zookeeper Quorum (see
> hbase-env.sh)
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
> <description>Comma separated list of servers in the ZooKeeper Quorum.
> For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
> By default this is set to localhost for local and pseudo-distributed
> modes
> of operation. For a fully-distributed setup, this should be set to a
> full
> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> hbase-env.sh
> this is the list of servers which we will start/stop ZooKeeper on.
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.property.tickTime</name>
> <value>1</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> The number of milliseconds of each tick. See
> zookeeper.session.timeout description.
> </description>
> </property>
> <property>
> <name>zookeeper.retries</name>
> <value>5</value>
> <description>How many times to retry connections to ZooKeeper. Used for
> reading/writing root region location, checking/writing out of safe
> mode.
> Used together with ${zookeeper.pause} in an exponential backoff
> fashion
> when making queries to ZooKeeper.
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.property.clientPort</name>
> <value>2222</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> the port at which the clients will connect.
> </description>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>3</value>
> <description>Default block replication.
> The actual number of replications can be specified when the file is created.
> The default is used if replication is not specified in create time.
> </description>
> </property>
> </configuration>
>
>
> The hbase-site.xml in slave1, slave2
> <configuration>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://Hadoopserver:54310/hbase</value>
> <description>The directory shared by region servers.
> Should be fully-qualified to include the filesystem to use.
> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> </description>
> </property>
> <property>
> <name>hbase.cluster.distributed</name>
> <value>true</value>
> <description>The mode the cluster will be in. Possible values are
> false: standalone and pseudo-distributed setups with managed Zookeeper
> true: fully-distributed with unmanaged Zookeeper Quorum (see
> hbase-env.sh)
> </description>
> </property>
> <property>
> <name>hbase.zookeeper.quorum</name>
> <value>Hadoopserver,Hadoopclient1,Hadoopclient</value>
> <description>Comma separated list of servers in the ZooKeeper Quorum.
> For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com".
> By default this is set to localhost for local and pseudo-distributed
> modes
> of operation. For a fully-distributed setup, this should be set to a
> full
> list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> hbase-env.sh
> this is the list of servers which we will start/stop ZooKeeper on.
> </description>
> </property>
>
> <property>
> <name>hbase.zookeeper.property.clientPort</name>
> <value>2222</value>
> <description>Property from ZooKeeper's config zoo.cfg.
> the port at which the clients will connect.
> </description>
> </property>
>
> <property>
> <name>dfs.replication</name>
> <value>3</value>
> <description>Default block replication.
> The actual number of replications can be specified when the file is created.
> The default is used if replication is not specified in create time.
> </description>
> </property>
> </configuration>
>
> Regionservers ( only on Master in my case Hadoopserver)
>
> Hadoopserver
> Hadoopclient1
> Hadoopclient
>
>
> Regionservers ( on slaves)
> localhost
>
> Blocked with this error for the past one week. Googled so much didnt get any
> solution.
>
>
> REgs.
> senthil
>