You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Yair Even-Zohar <ya...@audiencescience.com> on 2009/08/02 20:57:40 UTC
problem starting Hbase 0.20 with Katta on EC2
I have the following problem when trying to run hbase 0.20 on ec2.
First, I start katta which in turn starts zookeeper with the following
properties (zookeeper.zk.properties):
zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
#zookeeper client timeout in milliseconds
zookeeper.timeout=150000
#zookeeper tick time
zookeeper.tick-time=60000
# zookeeper init time limit
zookeeper.init-limit=5
# zookeeper sync limit
zookeeper.sync-limit=2
# zookeeper folder where data are stored
zookeeper.data-dir=/mnt/katta/data
# zookeeper folder where log data are stored
zookeeper.log-data-dir=/mnt/katta/logs
# zookeeper client port
zookeeper.clientPort=2181
I am running Hbase 0.20 RC1 with the following zk properties in
hbase-site.xml:
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>domU-12-31-39-06-29-91.compute-1.internal</value>
</property>
Unfortunately, I'm getting the following errors:
2009-08-02 14:32:09,381 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, host=domU-12-31-39-06-29-91.compute-1.internal:2181
sessionTimeout=30000
watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@120bf2c
2009-08-02 14:32:09,383 INFO org.apache.zookeeper.ClientCnxn:
zookeeper.disableAutoWatchReset is false
2009-08-02 14:32:09,415 INFO org.apache.zookeeper.ClientCnxn: Attempting
connection to server
domU-12-31-39-06-29-91.compute-1.internal/10.208.46.95:2181
2009-08-02 14:32:09,424 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@181ed9e
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:885)
2009-08-02 14:32:09,431 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown input
java.nio.channels.ClosedChannelException
at
sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
at
sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:951)
Looks like I have problem connecting to zookeeper but I have no clue
why. Server name is correct, client port is 2181 and user is root.
Any help?
Thanks
-Yair
RE: problem starting Hbase 0.20 with Katta on EC2
Posted by Yair Even-Zohar <ya...@audiencescience.com>.
Nope
I enabled the ports by running:
ec2-authorize GROUPID -p 3888
ec2-authorize GROUPID -p 2888
ec2-authorize GROUPID -p 2181
but I'm getting the exact same connection error
-----Original Message-----
From: Vaibhav Puranik [mailto:vpuranik@gmail.com]
Sent: Sunday, August 02, 2009 10:56 PM
To: hbase-user@hadoop.apache.org
Subject: Re: problem starting Hbase 0.20 with Katta on EC2
He means the EC2 security group, not ZK security groups.
EC2 comes with a concept of security groups. Every time you launch an
EC2
instance you can make it part of one or more security groups.
Ports on launched EC2 instances are not open by default unless you open
them
by configuring security groups. You can google for more information or
read
here - http://wiki.github.com/auser/poolparty/ec2-security-groups
Regards,
Vaibhav
On Sun, Aug 2, 2009 at 12:49 PM, Yair Even-Zohar
<ya...@audiencescience.com>wrote:
> What is ZK security group? (I know very little of zookeeper)
>
> I thought the following zookeeper props set these up
>
> zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
> zookeeper.clientPort=2181
>
> do I have to set them up in Hbase too (even though that's the
default)?
>
>
> -----Original Message-----
> From: Andrew Purtell [mailto:apurtell@apache.org]
> Sent: Sunday, August 02, 2009 10:13 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: problem starting Hbase 0.20 with Katta on EC2
>
> Hi,
>
> Have you set up the security group for your instances to allow the
> Zookeeper quorum peers to communicate with each other and clients?
>
> You need to allow all instances to communicate with each other on the
> following ports:
>
> 2181: client
> 2888: replication
> 3888: leader election
>
>
> The above are the defaults. All can be changed via various settings
> in hbase-site.xml (see hbase-default.xml).
>
> - Andy
>
>
>
> ________________________________
> From: Yair Even-Zohar <ya...@audiencescience.com>
> To: hbase-user@hadoop.apache.org;
katta-developer@lists.sourceforge.net
> Sent: Sunday, August 2, 2009 11:57:40 AM
> Subject: problem starting Hbase 0.20 with Katta on EC2
>
> I have the following problem when trying to run hbase 0.20 on ec2.
>
> First, I start katta which in turn starts zookeeper with the following
> properties (zookeeper.zk.properties):
>
>
>
> zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
>
> #zookeeper client timeout in milliseconds
>
> zookeeper.timeout=150000
>
> #zookeeper tick time
>
> zookeeper.tick-time=60000
>
> # zookeeper init time limit
>
> zookeeper.init-limit=5
>
> # zookeeper sync limit
>
> zookeeper.sync-limit=2
>
> # zookeeper folder where data are stored
>
> zookeeper.data-dir=/mnt/katta/data
>
> # zookeeper folder where log data are stored
>
> zookeeper.log-data-dir=/mnt/katta/logs
>
> # zookeeper client port
>
> zookeeper.clientPort=2181
>
>
>
> I am running Hbase 0.20 RC1 with the following zk properties in
> hbase-site.xml:
>
> <property>
>
> <name>hbase.cluster.distributed</name>
>
> <value>true</value>
>
> </property>
>
>
>
> <property>
>
> <name>hbase.zookeeper.quorum</name>
>
> <value>domU-12-31-39-06-29-91.compute-1.internal</value>
>
> </property>
>
>
>
>
>
>
>
> Unfortunately, I'm getting the following errors:
>
>
>
>
>
> 2009-08-02 14:32:09,381 INFO org.apache.zookeeper.ZooKeeper:
Initiating
> client connection, host=domU-12-31-39-06-29-91.compute-1.internal:2181
> sessionTimeout=30000
> watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@120bf2c
>
> 2009-08-02 14:32:09,383 INFO org.apache.zookeeper.ClientCnxn:
> zookeeper.disableAutoWatchReset is false
>
> 2009-08-02 14:32:09,415 INFO org.apache.zookeeper.ClientCnxn:
Attempting
> connection to server
> domU-12-31-39-06-29-91.compute-1.internal/10.208.46.95:2181
>
> 2009-08-02 14:32:09,424 WARN org.apache.zookeeper.ClientCnxn:
Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@181ed9e
>
> java.net.ConnectException: Connection refused
>
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:885)
>
> 2009-08-02 14:32:09,431 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
>
> java.nio.channels.ClosedChannelException
>
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>
> at
> sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>
> at
>
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:951)
>
>
>
> Looks like I have problem connecting to zookeeper but I have no clue
> why. Server name is correct, client port is 2181 and user is root.
>
> Any help?
>
>
>
> Thanks
>
> -Yair
>
>
>
>
Re: problem starting Hbase 0.20 with Katta on EC2
Posted by Vaibhav Puranik <vp...@gmail.com>.
He means the EC2 security group, not ZK security groups.
EC2 comes with a concept of security groups. Every time you launch an EC2
instance you can make it part of one or more security groups.
Ports on launched EC2 instances are not open by default unless you open them
by configuring security groups. You can google for more information or read
here - http://wiki.github.com/auser/poolparty/ec2-security-groups
Regards,
Vaibhav
On Sun, Aug 2, 2009 at 12:49 PM, Yair Even-Zohar
<ya...@audiencescience.com>wrote:
> What is ZK security group? (I know very little of zookeeper)
>
> I thought the following zookeeper props set these up
>
> zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
> zookeeper.clientPort=2181
>
> do I have to set them up in Hbase too (even though that's the default)?
>
>
> -----Original Message-----
> From: Andrew Purtell [mailto:apurtell@apache.org]
> Sent: Sunday, August 02, 2009 10:13 PM
> To: hbase-user@hadoop.apache.org
> Subject: Re: problem starting Hbase 0.20 with Katta on EC2
>
> Hi,
>
> Have you set up the security group for your instances to allow the
> Zookeeper quorum peers to communicate with each other and clients?
>
> You need to allow all instances to communicate with each other on the
> following ports:
>
> 2181: client
> 2888: replication
> 3888: leader election
>
>
> The above are the defaults. All can be changed via various settings
> in hbase-site.xml (see hbase-default.xml).
>
> - Andy
>
>
>
> ________________________________
> From: Yair Even-Zohar <ya...@audiencescience.com>
> To: hbase-user@hadoop.apache.org; katta-developer@lists.sourceforge.net
> Sent: Sunday, August 2, 2009 11:57:40 AM
> Subject: problem starting Hbase 0.20 with Katta on EC2
>
> I have the following problem when trying to run hbase 0.20 on ec2.
>
> First, I start katta which in turn starts zookeeper with the following
> properties (zookeeper.zk.properties):
>
>
>
> zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
>
> #zookeeper client timeout in milliseconds
>
> zookeeper.timeout=150000
>
> #zookeeper tick time
>
> zookeeper.tick-time=60000
>
> # zookeeper init time limit
>
> zookeeper.init-limit=5
>
> # zookeeper sync limit
>
> zookeeper.sync-limit=2
>
> # zookeeper folder where data are stored
>
> zookeeper.data-dir=/mnt/katta/data
>
> # zookeeper folder where log data are stored
>
> zookeeper.log-data-dir=/mnt/katta/logs
>
> # zookeeper client port
>
> zookeeper.clientPort=2181
>
>
>
> I am running Hbase 0.20 RC1 with the following zk properties in
> hbase-site.xml:
>
> <property>
>
> <name>hbase.cluster.distributed</name>
>
> <value>true</value>
>
> </property>
>
>
>
> <property>
>
> <name>hbase.zookeeper.quorum</name>
>
> <value>domU-12-31-39-06-29-91.compute-1.internal</value>
>
> </property>
>
>
>
>
>
>
>
> Unfortunately, I'm getting the following errors:
>
>
>
>
>
> 2009-08-02 14:32:09,381 INFO org.apache.zookeeper.ZooKeeper: Initiating
> client connection, host=domU-12-31-39-06-29-91.compute-1.internal:2181
> sessionTimeout=30000
> watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@120bf2c
>
> 2009-08-02 14:32:09,383 INFO org.apache.zookeeper.ClientCnxn:
> zookeeper.disableAutoWatchReset is false
>
> 2009-08-02 14:32:09,415 INFO org.apache.zookeeper.ClientCnxn: Attempting
> connection to server
> domU-12-31-39-06-29-91.compute-1.internal/10.208.46.95:2181
>
> 2009-08-02 14:32:09,424 WARN org.apache.zookeeper.ClientCnxn: Exception
> closing session 0x0 to sun.nio.ch.SelectionKeyImpl@181ed9e
>
> java.net.ConnectException: Connection refused
>
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>
> at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:885)
>
> 2009-08-02 14:32:09,431 WARN org.apache.zookeeper.ClientCnxn: Ignoring
> exception during shutdown input
>
> java.nio.channels.ClosedChannelException
>
> at
> sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
>
> at
> sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
>
> at
> org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:951)
>
>
>
> Looks like I have problem connecting to zookeeper but I have no clue
> why. Server name is correct, client port is 2181 and user is root.
>
> Any help?
>
>
>
> Thanks
>
> -Yair
>
>
>
>
RE: problem starting Hbase 0.20 with Katta on EC2
Posted by Yair Even-Zohar <ya...@audiencescience.com>.
What is ZK security group? (I know very little of zookeeper)
I thought the following zookeeper props set these up
zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
zookeeper.clientPort=2181
do I have to set them up in Hbase too (even though that's the default)?
-----Original Message-----
From: Andrew Purtell [mailto:apurtell@apache.org]
Sent: Sunday, August 02, 2009 10:13 PM
To: hbase-user@hadoop.apache.org
Subject: Re: problem starting Hbase 0.20 with Katta on EC2
Hi,
Have you set up the security group for your instances to allow the
Zookeeper quorum peers to communicate with each other and clients?
You need to allow all instances to communicate with each other on the
following ports:
2181: client
2888: replication
3888: leader election
The above are the defaults. All can be changed via various settings
in hbase-site.xml (see hbase-default.xml).
- Andy
________________________________
From: Yair Even-Zohar <ya...@audiencescience.com>
To: hbase-user@hadoop.apache.org; katta-developer@lists.sourceforge.net
Sent: Sunday, August 2, 2009 11:57:40 AM
Subject: problem starting Hbase 0.20 with Katta on EC2
I have the following problem when trying to run hbase 0.20 on ec2.
First, I start katta which in turn starts zookeeper with the following
properties (zookeeper.zk.properties):
zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
#zookeeper client timeout in milliseconds
zookeeper.timeout=150000
#zookeeper tick time
zookeeper.tick-time=60000
# zookeeper init time limit
zookeeper.init-limit=5
# zookeeper sync limit
zookeeper.sync-limit=2
# zookeeper folder where data are stored
zookeeper.data-dir=/mnt/katta/data
# zookeeper folder where log data are stored
zookeeper.log-data-dir=/mnt/katta/logs
# zookeeper client port
zookeeper.clientPort=2181
I am running Hbase 0.20 RC1 with the following zk properties in
hbase-site.xml:
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>domU-12-31-39-06-29-91.compute-1.internal</value>
</property>
Unfortunately, I'm getting the following errors:
2009-08-02 14:32:09,381 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, host=domU-12-31-39-06-29-91.compute-1.internal:2181
sessionTimeout=30000
watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@120bf2c
2009-08-02 14:32:09,383 INFO org.apache.zookeeper.ClientCnxn:
zookeeper.disableAutoWatchReset is false
2009-08-02 14:32:09,415 INFO org.apache.zookeeper.ClientCnxn: Attempting
connection to server
domU-12-31-39-06-29-91.compute-1.internal/10.208.46.95:2181
2009-08-02 14:32:09,424 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@181ed9e
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:885)
2009-08-02 14:32:09,431 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown input
java.nio.channels.ClosedChannelException
at
sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
at
sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:951)
Looks like I have problem connecting to zookeeper but I have no clue
why. Server name is correct, client port is 2181 and user is root.
Any help?
Thanks
-Yair
Re: problem starting Hbase 0.20 with Katta on EC2
Posted by Andrew Purtell <ap...@apache.org>.
Hi,
Have you set up the security group for your instances to allow the
Zookeeper quorum peers to communicate with each other and clients?
You need to allow all instances to communicate with each other on the
following ports:
2181: client
2888: replication
3888: leader election
The above are the defaults. All can be changed via various settings
in hbase-site.xml (see hbase-default.xml).
- Andy
________________________________
From: Yair Even-Zohar <ya...@audiencescience.com>
To: hbase-user@hadoop.apache.org; katta-developer@lists.sourceforge.net
Sent: Sunday, August 2, 2009 11:57:40 AM
Subject: problem starting Hbase 0.20 with Katta on EC2
I have the following problem when trying to run hbase 0.20 on ec2.
First, I start katta which in turn starts zookeeper with the following
properties (zookeeper.zk.properties):
zookeeper.servers=domU-12-31-39-06-29-91.compute-1.internal:2888:3888
#zookeeper client timeout in milliseconds
zookeeper.timeout=150000
#zookeeper tick time
zookeeper.tick-time=60000
# zookeeper init time limit
zookeeper.init-limit=5
# zookeeper sync limit
zookeeper.sync-limit=2
# zookeeper folder where data are stored
zookeeper.data-dir=/mnt/katta/data
# zookeeper folder where log data are stored
zookeeper.log-data-dir=/mnt/katta/logs
# zookeeper client port
zookeeper.clientPort=2181
I am running Hbase 0.20 RC1 with the following zk properties in
hbase-site.xml:
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>domU-12-31-39-06-29-91.compute-1.internal</value>
</property>
Unfortunately, I'm getting the following errors:
2009-08-02 14:32:09,381 INFO org.apache.zookeeper.ZooKeeper: Initiating
client connection, host=domU-12-31-39-06-29-91.compute-1.internal:2181
sessionTimeout=30000
watcher=org.apache.hadoop.hbase.regionserver.HRegionServer@120bf2c
2009-08-02 14:32:09,383 INFO org.apache.zookeeper.ClientCnxn:
zookeeper.disableAutoWatchReset is false
2009-08-02 14:32:09,415 INFO org.apache.zookeeper.ClientCnxn: Attempting
connection to server
domU-12-31-39-06-29-91.compute-1.internal/10.208.46.95:2181
2009-08-02 14:32:09,424 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x0 to sun.nio.ch.SelectionKeyImpl@181ed9e
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:885)
2009-08-02 14:32:09,431 WARN org.apache.zookeeper.ClientCnxn: Ignoring
exception during shutdown input
java.nio.channels.ClosedChannelException
at
sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:638)
at
sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:360)
at
org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:951)
Looks like I have problem connecting to zookeeper but I have no clue
why. Server name is correct, client port is 2181 and user is root.
Any help?
Thanks
-Yair