You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Joseph Coleman <jo...@infinitecampus.com> on 2011/03/01 15:41:26 UTC
Zookeeper / Hbase
Question when setting up the zookeeper master node can I have just
zookeeper on that box or do I need to have Hbase and hadoop installed on
there as well? I am just doing a 3 node setup for an exercise before my
production gear gets here.. So I have a working HDFS cluster 1 master and
one slave just trying to setup Hbase and Zookeeper with The zookeeper
master also serving as the Hbase master and the on one HDFS slave also
being my Hbase slave..
Re: Zookeeper / Hbase
Posted by Mark Kerzner <ma...@gmail.com>.
Hey, Joseph, I am also interested in Avatar - and can one make it work :)
Please share if you can.
Mark
On Tue, Mar 1, 2011 at 12:23 PM, Joseph Coleman <
joe.coleman@infinitecampus.com> wrote:
> Thanks, I am only doing 3 servers total for running through the setup for
> comfort sakes before my production gear gets here. I am looking at a
> single Master HDFS server will do Avatar if I can figure it out. The have
> a 10 node data cluster for HDFS and Hbase and a 3 node cluster for
> Zookeeper. Because at some point we way me 20 plus data nodes by years
> end.. Or am I better off leaving Hbase to manage ZK? If so what point do I
> separate out ZK to be its own management piece.
>
>
>
>
>
>
> On 3/1/11 9:15 AM, "Lars George" <la...@gmail.com> wrote:
>
> >Hi Joseph,
> >
> >You are talking about a full distributed setup - just all with single
> >nodes? So your ZooKeeper is started and maintained by you as well
> >separately? If so, then sure you can run it on your own. Well, even
> >with HBase you can run this on your own using the supplied version
> >that comes with HBase, see http://hbase.apache.org/book.html#zookeeper
> >for all the gory details.
> >
> >Lars
> >
> >On Tue, Mar 1, 2011 at 3:41 PM, Joseph Coleman
> ><jo...@infinitecampus.com> wrote:
> >> Question when setting up the zookeeper master node can I have just
> >> zookeeper on that box or do I need to have Hbase and hadoop installed on
> >> there as well? I am just doing a 3 node setup for an exercise before my
> >> production gear gets here.. So I have a working HDFS cluster 1 master
> >>and
> >> one slave just trying to setup Hbase and Zookeeper with The zookeeper
> >> master also serving as the Hbase master and the on one HDFS slave also
> >> being my Hbase slave..
> >>
> >>
>
>
Re: Zookeeper / Hbase
Posted by Lars George <la...@gmail.com>.
Hi Joseph,
As Dave says, you could always use HBase to manage ZooKeeper. If you
need it for other things as well, and the AvatarNode is one of those,
then you have to make sure you set
HBASE_MANAGES_ZK=false
in the hbase-env.sh but still use the HBase scripts to start and stop
the stand-alone ZooKeeper ensemble, including running the nodes on
shared servers, such as the master etc.
Lars
On Tue, Mar 1, 2011 at 7:23 PM, Joseph Coleman
<jo...@infinitecampus.com> wrote:
> Thanks, I am only doing 3 servers total for running through the setup for
> comfort sakes before my production gear gets here. I am looking at a
> single Master HDFS server will do Avatar if I can figure it out. The have
> a 10 node data cluster for HDFS and Hbase and a 3 node cluster for
> Zookeeper. Because at some point we way me 20 plus data nodes by years
> end.. Or am I better off leaving Hbase to manage ZK? If so what point do I
> separate out ZK to be its own management piece.
>
>
>
>
>
>
> On 3/1/11 9:15 AM, "Lars George" <la...@gmail.com> wrote:
>
>>Hi Joseph,
>>
>>You are talking about a full distributed setup - just all with single
>>nodes? So your ZooKeeper is started and maintained by you as well
>>separately? If so, then sure you can run it on your own. Well, even
>>with HBase you can run this on your own using the supplied version
>>that comes with HBase, see http://hbase.apache.org/book.html#zookeeper
>>for all the gory details.
>>
>>Lars
>>
>>On Tue, Mar 1, 2011 at 3:41 PM, Joseph Coleman
>><jo...@infinitecampus.com> wrote:
>>> Question when setting up the zookeeper master node can I have just
>>> zookeeper on that box or do I need to have Hbase and hadoop installed on
>>> there as well? I am just doing a 3 node setup for an exercise before my
>>> production gear gets here.. So I have a working HDFS cluster 1 master
>>>and
>>> one slave just trying to setup Hbase and Zookeeper with The zookeeper
>>> master also serving as the Hbase master and the on one HDFS slave also
>>> being my Hbase slave..
>>>
>>>
>
>
RE: Zookeeper / Hbase
Posted by "Buttler, David" <bu...@llnl.gov>.
I really like the way HBase manages zookeeper. It seems much more intuitive to me than the native zookeeper configuration. For my cluster I use zookeeper for a couple of different tasks (like hbase, solr cloud, and other home grown things). I manage Zookeeper using a slightly different version of the hbase scripts. This allows me to start and stop hbase without affecting the zookeeper cluster -- but I do have to remember to stop hbase if I need to stop zookeeper.
In my opinion, HBase improves zookeeper. [although I would be happy to hear why I am missing something from the standard zookeeper config]
Dave
-----Original Message-----
From: Lars George [mailto:lars.george@gmail.com]
Sent: Tuesday, March 01, 2011 7:15 AM
To: user@hbase.apache.org
Subject: Re: Zookeeper / Hbase
Hi Joseph,
You are talking about a full distributed setup - just all with single
nodes? So your ZooKeeper is started and maintained by you as well
separately? If so, then sure you can run it on your own. Well, even
with HBase you can run this on your own using the supplied version
that comes with HBase, see http://hbase.apache.org/book.html#zookeeper
for all the gory details.
Lars
On Tue, Mar 1, 2011 at 3:41 PM, Joseph Coleman
<jo...@infinitecampus.com> wrote:
> Question when setting up the zookeeper master node can I have just
> zookeeper on that box or do I need to have Hbase and hadoop installed on
> there as well? I am just doing a 3 node setup for an exercise before my
> production gear gets here.. So I have a working HDFS cluster 1 master and
> one slave just trying to setup Hbase and Zookeeper with The zookeeper
> master also serving as the Hbase master and the on one HDFS slave also
> being my Hbase slave..
>
>
Re: Zookeeper / Hbase
Posted by Joseph Coleman <jo...@infinitecampus.com>.
Thanks, I am only doing 3 servers total for running through the setup for
comfort sakes before my production gear gets here. I am looking at a
single Master HDFS server will do Avatar if I can figure it out. The have
a 10 node data cluster for HDFS and Hbase and a 3 node cluster for
Zookeeper. Because at some point we way me 20 plus data nodes by years
end.. Or am I better off leaving Hbase to manage ZK? If so what point do I
separate out ZK to be its own management piece.
On 3/1/11 9:15 AM, "Lars George" <la...@gmail.com> wrote:
>Hi Joseph,
>
>You are talking about a full distributed setup - just all with single
>nodes? So your ZooKeeper is started and maintained by you as well
>separately? If so, then sure you can run it on your own. Well, even
>with HBase you can run this on your own using the supplied version
>that comes with HBase, see http://hbase.apache.org/book.html#zookeeper
>for all the gory details.
>
>Lars
>
>On Tue, Mar 1, 2011 at 3:41 PM, Joseph Coleman
><jo...@infinitecampus.com> wrote:
>> Question when setting up the zookeeper master node can I have just
>> zookeeper on that box or do I need to have Hbase and hadoop installed on
>> there as well? I am just doing a 3 node setup for an exercise before my
>> production gear gets here.. So I have a working HDFS cluster 1 master
>>and
>> one slave just trying to setup Hbase and Zookeeper with The zookeeper
>> master also serving as the Hbase master and the on one HDFS slave also
>> being my Hbase slave..
>>
>>
Re: Zookeeper / Hbase
Posted by Lars George <la...@gmail.com>.
Hi Joseph,
You are talking about a full distributed setup - just all with single
nodes? So your ZooKeeper is started and maintained by you as well
separately? If so, then sure you can run it on your own. Well, even
with HBase you can run this on your own using the supplied version
that comes with HBase, see http://hbase.apache.org/book.html#zookeeper
for all the gory details.
Lars
On Tue, Mar 1, 2011 at 3:41 PM, Joseph Coleman
<jo...@infinitecampus.com> wrote:
> Question when setting up the zookeeper master node can I have just
> zookeeper on that box or do I need to have Hbase and hadoop installed on
> there as well? I am just doing a 3 node setup for an exercise before my
> production gear gets here.. So I have a working HDFS cluster 1 master and
> one slave just trying to setup Hbase and Zookeeper with The zookeeper
> master also serving as the Hbase master and the on one HDFS slave also
> being my Hbase slave..
>
>