API for node entry to the cluster.

[Avinash Lakshman <av...@gmail.com>]

Hi All

Is it possible to remove nodes and add nodes dynamically to the ZK cluster
via API? Any plans in the future to do this?

TIA
A

Re: API for node entry to the cluster.

[Avinash Lakshman <av...@gmail.com>]

Ah. Sorry. But I got it. I guess what Henry is working on is what I am
looking for.

Thanks
A

On Thu, Nov 5, 2009 at 11:32 AM, Ted Dunning <te...@gmail.com> wrote:

> You have two options.
>
> What Henry is talking about is a sophisticated update to ZK internals that
> does a very nice job of handling all the possible split-brain issues and
> other pathologies that can result from growing or shrinking a cluster.  His
> patch isn't ready yet, but when it is, it should be really nice.
>
> What I am talking about is pretty tame hack that allows you to get 99% of
> the result needed with very little effort.  It can be done today.
>
> Restarting servers one at a time is conventionally used for minor releases
> of ZK and can be (ab)used to allow configuration changes.
>
> I don't quite follow what you are saying in your second and third sentences
> so I can't answer the question in your fourth.
>
> Can you clarify what you mean?  I halfway suspect you dropped a word
> somewhere.
>
> On Thu, Nov 5, 2009 at 11:24 AM, Avinash Lakshman <
> avinash.lakshman@gmail.com> wrote:
>
> > I would prefer not restarting. Start/Stop the new/old process and then
> > start
> > a round of consensus for adding/removing a machine. I guess if one can do
> > that then there is stopping of process required.  Am I missing something
> > here?
> >
> > A
> >
> > On Thu, Nov 5, 2009 at 11:14 AM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > > It is pretty easy to do by hand and is pretty easy to script (it was
> for
> > > us).
> > >
> > > The process for adding is
> > >
> > > a) configure and start a new node
> > >
> > > b) re-configure and restart each existing node in turn to know about
> the
> > > new
> > > node
> > >
> > > you now have a larger cluster.
> > >
> > > To drop a node,
> > >
> > > a) reconfigure and restart each surviving node
> > >
> > > b) kill the node(s) that is(are) leaving
> > >
> > > you now have a smaller cluster.
> > >
> > > On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman <
> > > avinash.lakshman@gmail.com> wrote:
> > >
> > > > Hi All
> > > >
> > > > Is it possible to remove nodes and add nodes dynamically to the ZK
> > > cluster
> > > > via API? Any plans in the future to do this?
> > > >
> > > > TIA
> > > > A
> > > >
> > >
> > >
> > >
> > > --
> > > Ted Dunning, CTO
> > > DeepDyve
> > >
> >
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>

Re: API for node entry to the cluster.

[Ted Dunning <te...@gmail.com>]

You have two options.

What Henry is talking about is a sophisticated update to ZK internals that
does a very nice job of handling all the possible split-brain issues and
other pathologies that can result from growing or shrinking a cluster.  His
patch isn't ready yet, but when it is, it should be really nice.

What I am talking about is pretty tame hack that allows you to get 99% of
the result needed with very little effort.  It can be done today.

Restarting servers one at a time is conventionally used for minor releases
of ZK and can be (ab)used to allow configuration changes.

I don't quite follow what you are saying in your second and third sentences
so I can't answer the question in your fourth.

Can you clarify what you mean?  I halfway suspect you dropped a word
somewhere.

On Thu, Nov 5, 2009 at 11:24 AM, Avinash Lakshman <
avinash.lakshman@gmail.com> wrote:

> I would prefer not restarting. Start/Stop the new/old process and then
> start
> a round of consensus for adding/removing a machine. I guess if one can do
> that then there is stopping of process required.  Am I missing something
> here?
>
> A
>
> On Thu, Nov 5, 2009 at 11:14 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > It is pretty easy to do by hand and is pretty easy to script (it was for
> > us).
> >
> > The process for adding is
> >
> > a) configure and start a new node
> >
> > b) re-configure and restart each existing node in turn to know about the
> > new
> > node
> >
> > you now have a larger cluster.
> >
> > To drop a node,
> >
> > a) reconfigure and restart each surviving node
> >
> > b) kill the node(s) that is(are) leaving
> >
> > you now have a smaller cluster.
> >
> > On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman <
> > avinash.lakshman@gmail.com> wrote:
> >
> > > Hi All
> > >
> > > Is it possible to remove nodes and add nodes dynamically to the ZK
> > cluster
> > > via API? Any plans in the future to do this?
> > >
> > > TIA
> > > A
> > >
> >
> >
> >
> > --
> > Ted Dunning, CTO
> > DeepDyve
> >
>



-- 
Ted Dunning, CTO
DeepDyve

Re: API for node entry to the cluster.

[Patrick Hunt <ph...@apache.org>]

FYI - in Ted's scenario your application will most likely not be 
effected (if that's what you mean by "start a round of consensus").

 From your app's perspective your ZK clients will get disconnected from 
one server and reconnected to another (as the server it is connected to 
is restarted with the new config). If you are using something like our 
standard recipes, say leadership election, your ephemeral znodes and 
watches are uneffected - you don't see a new leadership election, 
everything continues as normal.

Actually you could shut down the entire ZK cluster and restart it, your 
ZK clients should be uneffected - they will reconnect to a server 
automatically when the cluster comes up and all ephemeral znodes and 
watches are maintained as they were prior to the restart.

Hope this helps.

Patrick

Avinash Lakshman wrote:
> I would prefer not restarting. Start/Stop the new/old process and then start
> a round of consensus for adding/removing a machine. I guess if one can do
> that then there is stopping of process required.  Am I missing something
> here?
> 
> A
> 
> On Thu, Nov 5, 2009 at 11:14 AM, Ted Dunning <te...@gmail.com> wrote:
> 
>> It is pretty easy to do by hand and is pretty easy to script (it was for
>> us).
>>
>> The process for adding is
>>
>> a) configure and start a new node
>>
>> b) re-configure and restart each existing node in turn to know about the
>> new
>> node
>>
>> you now have a larger cluster.
>>
>> To drop a node,
>>
>> a) reconfigure and restart each surviving node
>>
>> b) kill the node(s) that is(are) leaving
>>
>> you now have a smaller cluster.
>>
>> On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman <
>> avinash.lakshman@gmail.com> wrote:
>>
>>> Hi All
>>>
>>> Is it possible to remove nodes and add nodes dynamically to the ZK
>> cluster
>>> via API? Any plans in the future to do this?
>>>
>>> TIA
>>> A
>>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
> 

Re: API for node entry to the cluster.

[Avinash Lakshman <av...@gmail.com>]

I would prefer not restarting. Start/Stop the new/old process and then start
a round of consensus for adding/removing a machine. I guess if one can do
that then there is stopping of process required.  Am I missing something
here?

A

On Thu, Nov 5, 2009 at 11:14 AM, Ted Dunning <te...@gmail.com> wrote:

> It is pretty easy to do by hand and is pretty easy to script (it was for
> us).
>
> The process for adding is
>
> a) configure and start a new node
>
> b) re-configure and restart each existing node in turn to know about the
> new
> node
>
> you now have a larger cluster.
>
> To drop a node,
>
> a) reconfigure and restart each surviving node
>
> b) kill the node(s) that is(are) leaving
>
> you now have a smaller cluster.
>
> On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman <
> avinash.lakshman@gmail.com> wrote:
>
> > Hi All
> >
> > Is it possible to remove nodes and add nodes dynamically to the ZK
> cluster
> > via API? Any plans in the future to do this?
> >
> > TIA
> > A
> >
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>

Re: API for node entry to the cluster.

[Ted Dunning <te...@gmail.com>]

It is pretty easy to do by hand and is pretty easy to script (it was for
us).

The process for adding is

a) configure and start a new node

b) re-configure and restart each existing node in turn to know about the new
node

you now have a larger cluster.

To drop a node,

a) reconfigure and restart each surviving node

b) kill the node(s) that is(are) leaving

you now have a smaller cluster.

On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman <
avinash.lakshman@gmail.com> wrote:

> Hi All
>
> Is it possible to remove nodes and add nodes dynamically to the ZK cluster
> via API? Any plans in the future to do this?
>
> TIA
> A
>



-- 
Ted Dunning, CTO
DeepDyve

Re: API for node entry to the cluster.

[Henry Robinson <he...@cloudera.com>]

Hi -

Yes there are future plans. See
https://issues.apache.org/jira/browse/ZOOKEEPER-107. I have code written for
this that works but is not rock-solid yet.

cheers,
Henry

On Thu, Nov 5, 2009 at 11:02 AM, Avinash Lakshman <
avinash.lakshman@gmail.com> wrote:

> Hi All
>
> Is it possible to remove nodes and add nodes dynamically to the ZK cluster
> via API? Any plans in the future to do this?
>
> TIA
> A
>