ZKClient

[Jonathan Holloway <jo...@gmail.com>]

I came across this project on Github

http://github.com/sgroschupf/zkclient

for working with the Zookeeper API.  Has anybody used it in the past?  Is it
a better way of interacting with
a Zookeeper cluster?

Many thanks,
Jon.

Re: ZKClient

[Patrick Hunt <ph...@apache.org>]

While I agree DS is hard, I don't think we should lose the useful 
feedback given by Jonathan/Adam - that getting started with ZK is 
challenging and can be frustrating. We need to learn from this feedback 
and create some action items to address. One of the main things I've 
heard so far that we can act on today is that we should add 
examples/docs to round things out. I agree with this. Also the recipes 
page should be updated to point to the recipe implementations we 
recently added to the release.

One suggestion, it's much easier for new contributors/users to 
contribute to the examples than it is to jump into ZK core development. 
New users feel the pain most directly (recently), I'd encourage you to 
contribute back by creating an example or two. I'm sure the existing 
contributors would be happy to work with you to get them committed and 
released.

Regards,

Patrick

On 05/04/2010 03:43 PM, Ted Dunning wrote:
> Creating recipes is a great thing, but that doesn't change the fact that
> distributed systems are inherently a bit tricky, especially if you start
> with the assumption (as many people do) that Peter Deutsch was wrong.
>
> One of the great contributions of MapReduce style parallelism or the java
> concurrent package is that it provides safe trails in a pretty scary forest.
>   Good Zookeeper recipes could provide similar guidance with similar positive
> effects.
>
> On Tue, May 4, 2010 at 3:24 PM, Adam Rosien<ad...@rosien.net>  wrote:
>
>> I'll check it out, but it is repeated in this list and on the web site
>> that it's not as easy as it seems. I just want to enumerate the
>> failure points and create abstractions to avoid them.
>>
>> .. Adam
>>
>> On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar<ma...@yahoo-inc.com>
>> wrote:
>>> Hi Adam,
>>>   I don't think zk is very very hard to get right. There are exmaples in
>>> src/recipes which implements locks/queues/others. There is ZOOKEEPER-22
>> to
>>> make it even more easier for application to use.
>>>
>>> Regarding re registration of watches, you can deifnitely write code and
>>> submit is as a part of well documented contrib module which lays out the
>>> assumptions/design of it. It could very well be useful for others. Its
>> just
>>> that folks havent had much time to focus on these areas as yet.
>>>
>>> Thanks
>>> mahadev
>>>
>>>
>>> On 5/4/10 2:58 PM, "Adam Rosien"<ad...@rosien.net>  wrote:
>>>
>>>> I use zkclient in my work at kaChing and I have mixed feelings about
>>>> it. On one hand it makes "easy things easy" which is great, but on the
>>>> other hand I very few ideas what assumptions it makes "under the
>>>> hood". I also dislike some of the design choices such as unchecked
>>>> exceptions, but that's neither here nor there. It would take some
>>>> extensive documentation work by the authors to really enumerate the
>>>> model and assumptions, but the project doesn't seem to be active
>>>> (either from it being adequate for its current users or just
>>>> inactive). I'm not sure I could derive the assumptions myself.
>>>>
>>>> I'm a bit frustrated that zk is "very, very hard to really get right".
>>>> At a project level, can't we create structures to avoid most of these
>>>> errors? Can there be a "standard model" with detailed assumptions and
>>>> implementations of all the recipes? How can we start this? Is there
>>>> something that makes this too hard?
>>>>
>>>> I feel like a recipe page is a big fail; wouldn't an example app that
>>>> uses locks and barriers be that much more compelling?
>>>>
>>>> For the common FAQ items like "you need to re-register the watch",
>>>> can't we just create code that implements this pattern? My goal is to
>>>> live up to the motto: a good API is impossible to use incorrectly.
>>>>
>>>> .. Adam
>>>>
>>>> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning<te...@gmail.com>
>> wrote:
>>>>> In general, writing this sort of layer on top of ZK is very, very hard
>> to
>>>>> get really right for general use.  In a simple use-case, you can
>> probably
>>>>> nail it but distributed systems are a Zoo, to coin a phrase.  The
>> problem is
>>>>> that you are fundamentally changing the metaphors in use so assumptions
>> can
>>>>> come unglued or be introduced pretty easily.
>>>>>
>>>>> One example of this is the fact that ZK watches *don't* fire for every
>>>>> change but when you write listener oriented code, you kind of expect
>> that
>>>>> they will.  That makes it really, really easy to introduce that
>> assumption
>>>>> in the heads of the programmer using the event listener library on top
>> of
>>>>> ZK.  Another example is how the atomic get content/set watch call works
>> in
>>>>> ZK is easy to violate in an event driven architecture because the
>> thread
>>>>> that watches ZK probably resets the watch.  If you assume that the
>> listener
>>>>> will read the data, then you have introduced a timing mismatch between
>> the
>>>>> read of the data and the resetting of the watch.  That might be OK or
>> it
>>>>> might not be.  The point is that these changes are subtle and tricky to
>> get
>>>>> exactly right.
>>>>>
>>>>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway<
>>>>> jonathan.holloway@gmail.com>  wrote:
>>>>>
>>>>>> Is there any reason why this isn't part of the Zookeeper trunk
>> already?
>>>>>>
>>>>>
>>>
>>>
>>
>

Re: ZKClient

[Patrick Hunt <ph...@apache.org>]

While I agree DS is hard, I don't think we should lose the useful 
feedback given by Jonathan/Adam - that getting started with ZK is 
challenging and can be frustrating. We need to learn from this feedback 
and create some action items to address. One of the main things I've 
heard so far that we can act on today is that we should add 
examples/docs to round things out. I agree with this. Also the recipes 
page should be updated to point to the recipe implementations we 
recently added to the release.

One suggestion, it's much easier for new contributors/users to 
contribute to the examples than it is to jump into ZK core development. 
New users feel the pain most directly (recently), I'd encourage you to 
contribute back by creating an example or two. I'm sure the existing 
contributors would be happy to work with you to get them committed and 
released.

Regards,

Patrick

On 05/04/2010 03:43 PM, Ted Dunning wrote:
> Creating recipes is a great thing, but that doesn't change the fact that
> distributed systems are inherently a bit tricky, especially if you start
> with the assumption (as many people do) that Peter Deutsch was wrong.
>
> One of the great contributions of MapReduce style parallelism or the java
> concurrent package is that it provides safe trails in a pretty scary forest.
>   Good Zookeeper recipes could provide similar guidance with similar positive
> effects.
>
> On Tue, May 4, 2010 at 3:24 PM, Adam Rosien<ad...@rosien.net>  wrote:
>
>> I'll check it out, but it is repeated in this list and on the web site
>> that it's not as easy as it seems. I just want to enumerate the
>> failure points and create abstractions to avoid them.
>>
>> .. Adam
>>
>> On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar<ma...@yahoo-inc.com>
>> wrote:
>>> Hi Adam,
>>>   I don't think zk is very very hard to get right. There are exmaples in
>>> src/recipes which implements locks/queues/others. There is ZOOKEEPER-22
>> to
>>> make it even more easier for application to use.
>>>
>>> Regarding re registration of watches, you can deifnitely write code and
>>> submit is as a part of well documented contrib module which lays out the
>>> assumptions/design of it. It could very well be useful for others. Its
>> just
>>> that folks havent had much time to focus on these areas as yet.
>>>
>>> Thanks
>>> mahadev
>>>
>>>
>>> On 5/4/10 2:58 PM, "Adam Rosien"<ad...@rosien.net>  wrote:
>>>
>>>> I use zkclient in my work at kaChing and I have mixed feelings about
>>>> it. On one hand it makes "easy things easy" which is great, but on the
>>>> other hand I very few ideas what assumptions it makes "under the
>>>> hood". I also dislike some of the design choices such as unchecked
>>>> exceptions, but that's neither here nor there. It would take some
>>>> extensive documentation work by the authors to really enumerate the
>>>> model and assumptions, but the project doesn't seem to be active
>>>> (either from it being adequate for its current users or just
>>>> inactive). I'm not sure I could derive the assumptions myself.
>>>>
>>>> I'm a bit frustrated that zk is "very, very hard to really get right".
>>>> At a project level, can't we create structures to avoid most of these
>>>> errors? Can there be a "standard model" with detailed assumptions and
>>>> implementations of all the recipes? How can we start this? Is there
>>>> something that makes this too hard?
>>>>
>>>> I feel like a recipe page is a big fail; wouldn't an example app that
>>>> uses locks and barriers be that much more compelling?
>>>>
>>>> For the common FAQ items like "you need to re-register the watch",
>>>> can't we just create code that implements this pattern? My goal is to
>>>> live up to the motto: a good API is impossible to use incorrectly.
>>>>
>>>> .. Adam
>>>>
>>>> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning<te...@gmail.com>
>> wrote:
>>>>> In general, writing this sort of layer on top of ZK is very, very hard
>> to
>>>>> get really right for general use.  In a simple use-case, you can
>> probably
>>>>> nail it but distributed systems are a Zoo, to coin a phrase.  The
>> problem is
>>>>> that you are fundamentally changing the metaphors in use so assumptions
>> can
>>>>> come unglued or be introduced pretty easily.
>>>>>
>>>>> One example of this is the fact that ZK watches *don't* fire for every
>>>>> change but when you write listener oriented code, you kind of expect
>> that
>>>>> they will.  That makes it really, really easy to introduce that
>> assumption
>>>>> in the heads of the programmer using the event listener library on top
>> of
>>>>> ZK.  Another example is how the atomic get content/set watch call works
>> in
>>>>> ZK is easy to violate in an event driven architecture because the
>> thread
>>>>> that watches ZK probably resets the watch.  If you assume that the
>> listener
>>>>> will read the data, then you have introduced a timing mismatch between
>> the
>>>>> read of the data and the resetting of the watch.  That might be OK or
>> it
>>>>> might not be.  The point is that these changes are subtle and tricky to
>> get
>>>>> exactly right.
>>>>>
>>>>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway<
>>>>> jonathan.holloway@gmail.com>  wrote:
>>>>>
>>>>>> Is there any reason why this isn't part of the Zookeeper trunk
>> already?
>>>>>>
>>>>>
>>>
>>>
>>
>

Re: ZKClient

[Ted Dunning <te...@gmail.com>]

Creating recipes is a great thing, but that doesn't change the fact that
distributed systems are inherently a bit tricky, especially if you start
with the assumption (as many people do) that Peter Deutsch was wrong.

One of the great contributions of MapReduce style parallelism or the java
concurrent package is that it provides safe trails in a pretty scary forest.
 Good Zookeeper recipes could provide similar guidance with similar positive
effects.

On Tue, May 4, 2010 at 3:24 PM, Adam Rosien <ad...@rosien.net> wrote:

> I'll check it out, but it is repeated in this list and on the web site
> that it's not as easy as it seems. I just want to enumerate the
> failure points and create abstractions to avoid them.
>
> .. Adam
>
> On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar <ma...@yahoo-inc.com>
> wrote:
> > Hi Adam,
> >  I don't think zk is very very hard to get right. There are exmaples in
> > src/recipes which implements locks/queues/others. There is ZOOKEEPER-22
> to
> > make it even more easier for application to use.
> >
> > Regarding re registration of watches, you can deifnitely write code and
> > submit is as a part of well documented contrib module which lays out the
> > assumptions/design of it. It could very well be useful for others. Its
> just
> > that folks havent had much time to focus on these areas as yet.
> >
> > Thanks
> > mahadev
> >
> >
> > On 5/4/10 2:58 PM, "Adam Rosien" <ad...@rosien.net> wrote:
> >
> >> I use zkclient in my work at kaChing and I have mixed feelings about
> >> it. On one hand it makes "easy things easy" which is great, but on the
> >> other hand I very few ideas what assumptions it makes "under the
> >> hood". I also dislike some of the design choices such as unchecked
> >> exceptions, but that's neither here nor there. It would take some
> >> extensive documentation work by the authors to really enumerate the
> >> model and assumptions, but the project doesn't seem to be active
> >> (either from it being adequate for its current users or just
> >> inactive). I'm not sure I could derive the assumptions myself.
> >>
> >> I'm a bit frustrated that zk is "very, very hard to really get right".
> >> At a project level, can't we create structures to avoid most of these
> >> errors? Can there be a "standard model" with detailed assumptions and
> >> implementations of all the recipes? How can we start this? Is there
> >> something that makes this too hard?
> >>
> >> I feel like a recipe page is a big fail; wouldn't an example app that
> >> uses locks and barriers be that much more compelling?
> >>
> >> For the common FAQ items like "you need to re-register the watch",
> >> can't we just create code that implements this pattern? My goal is to
> >> live up to the motto: a good API is impossible to use incorrectly.
> >>
> >> .. Adam
> >>
> >> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning <te...@gmail.com>
> wrote:
> >>> In general, writing this sort of layer on top of ZK is very, very hard
> to
> >>> get really right for general use.  In a simple use-case, you can
> probably
> >>> nail it but distributed systems are a Zoo, to coin a phrase.  The
> problem is
> >>> that you are fundamentally changing the metaphors in use so assumptions
> can
> >>> come unglued or be introduced pretty easily.
> >>>
> >>> One example of this is the fact that ZK watches *don't* fire for every
> >>> change but when you write listener oriented code, you kind of expect
> that
> >>> they will.  That makes it really, really easy to introduce that
> assumption
> >>> in the heads of the programmer using the event listener library on top
> of
> >>> ZK.  Another example is how the atomic get content/set watch call works
> in
> >>> ZK is easy to violate in an event driven architecture because the
> thread
> >>> that watches ZK probably resets the watch.  If you assume that the
> listener
> >>> will read the data, then you have introduced a timing mismatch between
> the
> >>> read of the data and the resetting of the watch.  That might be OK or
> it
> >>> might not be.  The point is that these changes are subtle and tricky to
> get
> >>> exactly right.
> >>>
> >>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
> >>> jonathan.holloway@gmail.com> wrote:
> >>>
> >>>> Is there any reason why this isn't part of the Zookeeper trunk
> already?
> >>>>
> >>>
> >
> >
>

Re: ZKClient

[Adam Rosien <ad...@rosien.net>]

I'll check it out, but it is repeated in this list and on the web site
that it's not as easy as it seems. I just want to enumerate the
failure points and create abstractions to avoid them.

.. Adam

On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar <ma...@yahoo-inc.com> wrote:
> Hi Adam,
>  I don't think zk is very very hard to get right. There are exmaples in
> src/recipes which implements locks/queues/others. There is ZOOKEEPER-22 to
> make it even more easier for application to use.
>
> Regarding re registration of watches, you can deifnitely write code and
> submit is as a part of well documented contrib module which lays out the
> assumptions/design of it. It could very well be useful for others. Its just
> that folks havent had much time to focus on these areas as yet.
>
> Thanks
> mahadev
>
>
> On 5/4/10 2:58 PM, "Adam Rosien" <ad...@rosien.net> wrote:
>
>> I use zkclient in my work at kaChing and I have mixed feelings about
>> it. On one hand it makes "easy things easy" which is great, but on the
>> other hand I very few ideas what assumptions it makes "under the
>> hood". I also dislike some of the design choices such as unchecked
>> exceptions, but that's neither here nor there. It would take some
>> extensive documentation work by the authors to really enumerate the
>> model and assumptions, but the project doesn't seem to be active
>> (either from it being adequate for its current users or just
>> inactive). I'm not sure I could derive the assumptions myself.
>>
>> I'm a bit frustrated that zk is "very, very hard to really get right".
>> At a project level, can't we create structures to avoid most of these
>> errors? Can there be a "standard model" with detailed assumptions and
>> implementations of all the recipes? How can we start this? Is there
>> something that makes this too hard?
>>
>> I feel like a recipe page is a big fail; wouldn't an example app that
>> uses locks and barriers be that much more compelling?
>>
>> For the common FAQ items like "you need to re-register the watch",
>> can't we just create code that implements this pattern? My goal is to
>> live up to the motto: a good API is impossible to use incorrectly.
>>
>> .. Adam
>>
>> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning <te...@gmail.com> wrote:
>>> In general, writing this sort of layer on top of ZK is very, very hard to
>>> get really right for general use.  In a simple use-case, you can probably
>>> nail it but distributed systems are a Zoo, to coin a phrase.  The problem is
>>> that you are fundamentally changing the metaphors in use so assumptions can
>>> come unglued or be introduced pretty easily.
>>>
>>> One example of this is the fact that ZK watches *don't* fire for every
>>> change but when you write listener oriented code, you kind of expect that
>>> they will.  That makes it really, really easy to introduce that assumption
>>> in the heads of the programmer using the event listener library on top of
>>> ZK.  Another example is how the atomic get content/set watch call works in
>>> ZK is easy to violate in an event driven architecture because the thread
>>> that watches ZK probably resets the watch.  If you assume that the listener
>>> will read the data, then you have introduced a timing mismatch between the
>>> read of the data and the resetting of the watch.  That might be OK or it
>>> might not be.  The point is that these changes are subtle and tricky to get
>>> exactly right.
>>>
>>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
>>> jonathan.holloway@gmail.com> wrote:
>>>
>>>> Is there any reason why this isn't part of the Zookeeper trunk already?
>>>>
>>>
>
>

Re: ZKClient

[Patrick Hunt <ph...@apache.org>]

Thanks Travis, I've slated this for 3.4.0, I think it would be useful to 
add more examples so feel free to add more if you have any ideas for 
useful ones.

For future reference, we ask that contributions come in the form of a patch:
http://wiki.apache.org/hadoop/ZooKeeper/HowToContribute

It's fine this time around, but in future it would be helpful.

(also click on "submit patch" link when you are ready for review - that 
pushes it through the process, incl automated testing/verification, 
that's why we ask for a patch off the root btw)

Thanks!

Patrick

On 05/04/2010 04:00 PM, Travis Crawford wrote:
> On Tue, May 4, 2010 at 3:45 PM, Ted Dunning<te...@gmail.com>  wrote:
>> Travis,
>>
>> Attachments are stripped from this mailing list.  Can you file a JIRA and
>> put your attachment on that instead?
>>
>> Here is a link to get you started:
>> https://issues.apache.org/jira/browse/ZOOKEEPER
>
> Whoops. Filed:
>
> https://issues.apache.org/jira/browse/ZOOKEEPER-765
>
> --travis
>
>
>>
>> On Tue, May 4, 2010 at 3:43 PM, Travis Crawford<tr...@gmail.com>wrote:
>>
>>> Attached is a skeleton application I extracted from a script I use --
>>> perhaps we could add this as a recipe? If there are issues I'm more
>>> than happy to fix them, or add more comments, whatever. It took a
>>> while to figure this out and I'd love to save others that time in the
>>> future.
>>>
>>> --travis
>>>
>>>
>>> On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar<ma...@yahoo-inc.com>
>>> wrote:
>>>> Hi Adam,
>>>>   I don't think zk is very very hard to get right. There are exmaples in
>>>> src/recipes which implements locks/queues/others. There is ZOOKEEPER-22
>>> to
>>>> make it even more easier for application to use.
>>>>
>>>> Regarding re registration of watches, you can deifnitely write code and
>>>> submit is as a part of well documented contrib module which lays out the
>>>> assumptions/design of it. It could very well be useful for others. Its
>>> just
>>>> that folks havent had much time to focus on these areas as yet.
>>>>
>>>> Thanks
>>>> mahadev
>>>>
>>>>
>>>> On 5/4/10 2:58 PM, "Adam Rosien"<ad...@rosien.net>  wrote:
>>>>
>>>>> I use zkclient in my work at kaChing and I have mixed feelings about
>>>>> it. On one hand it makes "easy things easy" which is great, but on the
>>>>> other hand I very few ideas what assumptions it makes "under the
>>>>> hood". I also dislike some of the design choices such as unchecked
>>>>> exceptions, but that's neither here nor there. It would take some
>>>>> extensive documentation work by the authors to really enumerate the
>>>>> model and assumptions, but the project doesn't seem to be active
>>>>> (either from it being adequate for its current users or just
>>>>> inactive). I'm not sure I could derive the assumptions myself.
>>>>>
>>>>> I'm a bit frustrated that zk is "very, very hard to really get right".
>>>>> At a project level, can't we create structures to avoid most of these
>>>>> errors? Can there be a "standard model" with detailed assumptions and
>>>>> implementations of all the recipes? How can we start this? Is there
>>>>> something that makes this too hard?
>>>>>
>>>>> I feel like a recipe page is a big fail; wouldn't an example app that
>>>>> uses locks and barriers be that much more compelling?
>>>>>
>>>>> For the common FAQ items like "you need to re-register the watch",
>>>>> can't we just create code that implements this pattern? My goal is to
>>>>> live up to the motto: a good API is impossible to use incorrectly.
>>>>>
>>>>> .. Adam
>>>>>
>>>>> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning<te...@gmail.com>
>>> wrote:
>>>>>> In general, writing this sort of layer on top of ZK is very, very hard
>>> to
>>>>>> get really right for general use.  In a simple use-case, you can
>>> probably
>>>>>> nail it but distributed systems are a Zoo, to coin a phrase.  The
>>> problem is
>>>>>> that you are fundamentally changing the metaphors in use so assumptions
>>> can
>>>>>> come unglued or be introduced pretty easily.
>>>>>>
>>>>>> One example of this is the fact that ZK watches *don't* fire for every
>>>>>> change but when you write listener oriented code, you kind of expect
>>> that
>>>>>> they will.  That makes it really, really easy to introduce that
>>> assumption
>>>>>> in the heads of the programmer using the event listener library on top
>>> of
>>>>>> ZK.  Another example is how the atomic get content/set watch call works
>>> in
>>>>>> ZK is easy to violate in an event driven architecture because the
>>> thread
>>>>>> that watches ZK probably resets the watch.  If you assume that the
>>> listener
>>>>>> will read the data, then you have introduced a timing mismatch between
>>> the
>>>>>> read of the data and the resetting of the watch.  That might be OK or
>>> it
>>>>>> might not be.  The point is that these changes are subtle and tricky to
>>> get
>>>>>> exactly right.
>>>>>>
>>>>>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway<
>>>>>> jonathan.holloway@gmail.com>  wrote:
>>>>>>
>>>>>>> Is there any reason why this isn't part of the Zookeeper trunk
>>> already?
>>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>

Re: ZKClient

[Travis Crawford <tr...@gmail.com>]

On Tue, May 4, 2010 at 3:45 PM, Ted Dunning <te...@gmail.com> wrote:
> Travis,
>
> Attachments are stripped from this mailing list.  Can you file a JIRA and
> put your attachment on that instead?
>
> Here is a link to get you started:
> https://issues.apache.org/jira/browse/ZOOKEEPER

Whoops. Filed:

https://issues.apache.org/jira/browse/ZOOKEEPER-765

--travis


>
> On Tue, May 4, 2010 at 3:43 PM, Travis Crawford <tr...@gmail.com>wrote:
>
>> Attached is a skeleton application I extracted from a script I use --
>> perhaps we could add this as a recipe? If there are issues I'm more
>> than happy to fix them, or add more comments, whatever. It took a
>> while to figure this out and I'd love to save others that time in the
>> future.
>>
>> --travis
>>
>>
>> On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar <ma...@yahoo-inc.com>
>> wrote:
>> > Hi Adam,
>> >  I don't think zk is very very hard to get right. There are exmaples in
>> > src/recipes which implements locks/queues/others. There is ZOOKEEPER-22
>> to
>> > make it even more easier for application to use.
>> >
>> > Regarding re registration of watches, you can deifnitely write code and
>> > submit is as a part of well documented contrib module which lays out the
>> > assumptions/design of it. It could very well be useful for others. Its
>> just
>> > that folks havent had much time to focus on these areas as yet.
>> >
>> > Thanks
>> > mahadev
>> >
>> >
>> > On 5/4/10 2:58 PM, "Adam Rosien" <ad...@rosien.net> wrote:
>> >
>> >> I use zkclient in my work at kaChing and I have mixed feelings about
>> >> it. On one hand it makes "easy things easy" which is great, but on the
>> >> other hand I very few ideas what assumptions it makes "under the
>> >> hood". I also dislike some of the design choices such as unchecked
>> >> exceptions, but that's neither here nor there. It would take some
>> >> extensive documentation work by the authors to really enumerate the
>> >> model and assumptions, but the project doesn't seem to be active
>> >> (either from it being adequate for its current users or just
>> >> inactive). I'm not sure I could derive the assumptions myself.
>> >>
>> >> I'm a bit frustrated that zk is "very, very hard to really get right".
>> >> At a project level, can't we create structures to avoid most of these
>> >> errors? Can there be a "standard model" with detailed assumptions and
>> >> implementations of all the recipes? How can we start this? Is there
>> >> something that makes this too hard?
>> >>
>> >> I feel like a recipe page is a big fail; wouldn't an example app that
>> >> uses locks and barriers be that much more compelling?
>> >>
>> >> For the common FAQ items like "you need to re-register the watch",
>> >> can't we just create code that implements this pattern? My goal is to
>> >> live up to the motto: a good API is impossible to use incorrectly.
>> >>
>> >> .. Adam
>> >>
>> >> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning <te...@gmail.com>
>> wrote:
>> >>> In general, writing this sort of layer on top of ZK is very, very hard
>> to
>> >>> get really right for general use.  In a simple use-case, you can
>> probably
>> >>> nail it but distributed systems are a Zoo, to coin a phrase.  The
>> problem is
>> >>> that you are fundamentally changing the metaphors in use so assumptions
>> can
>> >>> come unglued or be introduced pretty easily.
>> >>>
>> >>> One example of this is the fact that ZK watches *don't* fire for every
>> >>> change but when you write listener oriented code, you kind of expect
>> that
>> >>> they will.  That makes it really, really easy to introduce that
>> assumption
>> >>> in the heads of the programmer using the event listener library on top
>> of
>> >>> ZK.  Another example is how the atomic get content/set watch call works
>> in
>> >>> ZK is easy to violate in an event driven architecture because the
>> thread
>> >>> that watches ZK probably resets the watch.  If you assume that the
>> listener
>> >>> will read the data, then you have introduced a timing mismatch between
>> the
>> >>> read of the data and the resetting of the watch.  That might be OK or
>> it
>> >>> might not be.  The point is that these changes are subtle and tricky to
>> get
>> >>> exactly right.
>> >>>
>> >>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
>> >>> jonathan.holloway@gmail.com> wrote:
>> >>>
>> >>>> Is there any reason why this isn't part of the Zookeeper trunk
>> already?
>> >>>>
>> >>>
>> >
>> >
>>
>

Re: ZKClient

[Ted Dunning <te...@gmail.com>]

Travis,

Attachments are stripped from this mailing list.  Can you file a JIRA and
put your attachment on that instead?

Here is a link to get you started:
https://issues.apache.org/jira/browse/ZOOKEEPER

On Tue, May 4, 2010 at 3:43 PM, Travis Crawford <tr...@gmail.com>wrote:

> Attached is a skeleton application I extracted from a script I use --
> perhaps we could add this as a recipe? If there are issues I'm more
> than happy to fix them, or add more comments, whatever. It took a
> while to figure this out and I'd love to save others that time in the
> future.
>
> --travis
>
>
> On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar <ma...@yahoo-inc.com>
> wrote:
> > Hi Adam,
> >  I don't think zk is very very hard to get right. There are exmaples in
> > src/recipes which implements locks/queues/others. There is ZOOKEEPER-22
> to
> > make it even more easier for application to use.
> >
> > Regarding re registration of watches, you can deifnitely write code and
> > submit is as a part of well documented contrib module which lays out the
> > assumptions/design of it. It could very well be useful for others. Its
> just
> > that folks havent had much time to focus on these areas as yet.
> >
> > Thanks
> > mahadev
> >
> >
> > On 5/4/10 2:58 PM, "Adam Rosien" <ad...@rosien.net> wrote:
> >
> >> I use zkclient in my work at kaChing and I have mixed feelings about
> >> it. On one hand it makes "easy things easy" which is great, but on the
> >> other hand I very few ideas what assumptions it makes "under the
> >> hood". I also dislike some of the design choices such as unchecked
> >> exceptions, but that's neither here nor there. It would take some
> >> extensive documentation work by the authors to really enumerate the
> >> model and assumptions, but the project doesn't seem to be active
> >> (either from it being adequate for its current users or just
> >> inactive). I'm not sure I could derive the assumptions myself.
> >>
> >> I'm a bit frustrated that zk is "very, very hard to really get right".
> >> At a project level, can't we create structures to avoid most of these
> >> errors? Can there be a "standard model" with detailed assumptions and
> >> implementations of all the recipes? How can we start this? Is there
> >> something that makes this too hard?
> >>
> >> I feel like a recipe page is a big fail; wouldn't an example app that
> >> uses locks and barriers be that much more compelling?
> >>
> >> For the common FAQ items like "you need to re-register the watch",
> >> can't we just create code that implements this pattern? My goal is to
> >> live up to the motto: a good API is impossible to use incorrectly.
> >>
> >> .. Adam
> >>
> >> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning <te...@gmail.com>
> wrote:
> >>> In general, writing this sort of layer on top of ZK is very, very hard
> to
> >>> get really right for general use.  In a simple use-case, you can
> probably
> >>> nail it but distributed systems are a Zoo, to coin a phrase.  The
> problem is
> >>> that you are fundamentally changing the metaphors in use so assumptions
> can
> >>> come unglued or be introduced pretty easily.
> >>>
> >>> One example of this is the fact that ZK watches *don't* fire for every
> >>> change but when you write listener oriented code, you kind of expect
> that
> >>> they will.  That makes it really, really easy to introduce that
> assumption
> >>> in the heads of the programmer using the event listener library on top
> of
> >>> ZK.  Another example is how the atomic get content/set watch call works
> in
> >>> ZK is easy to violate in an event driven architecture because the
> thread
> >>> that watches ZK probably resets the watch.  If you assume that the
> listener
> >>> will read the data, then you have introduced a timing mismatch between
> the
> >>> read of the data and the resetting of the watch.  That might be OK or
> it
> >>> might not be.  The point is that these changes are subtle and tricky to
> get
> >>> exactly right.
> >>>
> >>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
> >>> jonathan.holloway@gmail.com> wrote:
> >>>
> >>>> Is there any reason why this isn't part of the Zookeeper trunk
> already?
> >>>>
> >>>
> >
> >
>

Re: ZKClient

[Travis Crawford <tr...@gmail.com>]

Attached is a skeleton application I extracted from a script I use --
perhaps we could add this as a recipe? If there are issues I'm more
than happy to fix them, or add more comments, whatever. It took a
while to figure this out and I'd love to save others that time in the
future.

--travis


On Tue, May 4, 2010 at 3:16 PM, Mahadev Konar <ma...@yahoo-inc.com> wrote:
> Hi Adam,
>  I don't think zk is very very hard to get right. There are exmaples in
> src/recipes which implements locks/queues/others. There is ZOOKEEPER-22 to
> make it even more easier for application to use.
>
> Regarding re registration of watches, you can deifnitely write code and
> submit is as a part of well documented contrib module which lays out the
> assumptions/design of it. It could very well be useful for others. Its just
> that folks havent had much time to focus on these areas as yet.
>
> Thanks
> mahadev
>
>
> On 5/4/10 2:58 PM, "Adam Rosien" <ad...@rosien.net> wrote:
>
>> I use zkclient in my work at kaChing and I have mixed feelings about
>> it. On one hand it makes "easy things easy" which is great, but on the
>> other hand I very few ideas what assumptions it makes "under the
>> hood". I also dislike some of the design choices such as unchecked
>> exceptions, but that's neither here nor there. It would take some
>> extensive documentation work by the authors to really enumerate the
>> model and assumptions, but the project doesn't seem to be active
>> (either from it being adequate for its current users or just
>> inactive). I'm not sure I could derive the assumptions myself.
>>
>> I'm a bit frustrated that zk is "very, very hard to really get right".
>> At a project level, can't we create structures to avoid most of these
>> errors? Can there be a "standard model" with detailed assumptions and
>> implementations of all the recipes? How can we start this? Is there
>> something that makes this too hard?
>>
>> I feel like a recipe page is a big fail; wouldn't an example app that
>> uses locks and barriers be that much more compelling?
>>
>> For the common FAQ items like "you need to re-register the watch",
>> can't we just create code that implements this pattern? My goal is to
>> live up to the motto: a good API is impossible to use incorrectly.
>>
>> .. Adam
>>
>> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning <te...@gmail.com> wrote:
>>> In general, writing this sort of layer on top of ZK is very, very hard to
>>> get really right for general use.  In a simple use-case, you can probably
>>> nail it but distributed systems are a Zoo, to coin a phrase.  The problem is
>>> that you are fundamentally changing the metaphors in use so assumptions can
>>> come unglued or be introduced pretty easily.
>>>
>>> One example of this is the fact that ZK watches *don't* fire for every
>>> change but when you write listener oriented code, you kind of expect that
>>> they will.  That makes it really, really easy to introduce that assumption
>>> in the heads of the programmer using the event listener library on top of
>>> ZK.  Another example is how the atomic get content/set watch call works in
>>> ZK is easy to violate in an event driven architecture because the thread
>>> that watches ZK probably resets the watch.  If you assume that the listener
>>> will read the data, then you have introduced a timing mismatch between the
>>> read of the data and the resetting of the watch.  That might be OK or it
>>> might not be.  The point is that these changes are subtle and tricky to get
>>> exactly right.
>>>
>>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
>>> jonathan.holloway@gmail.com> wrote:
>>>
>>>> Is there any reason why this isn't part of the Zookeeper trunk already?
>>>>
>>>
>
>

Re: ZKClient

[Ted Dunning <te...@gmail.com>]

I don't think that zk is hard to get right.

What is hard is to layer a very different model on top of ZK that changes
the semantics significantly and that that translation right.

One of the very cool things about ZK is how easy it is to write correct
code.  I know that Ben and co put a lot of thought into making sure that the
API would encourage correct behavior.

I am very sorry if my words suggest anything different.

On Tue, May 4, 2010 at 2:58 PM, Adam Rosien <ad...@rosien.net> wrote:

> I'm a bit frustrated that zk is "very, very hard to really get right".
> At a project level, can't we create structures to avoid most of these
> errors? Can there be a "standard model" with detailed assumptions and
> implementations of all the recipes? How can we start this? Is there
> something that makes this too hard?
>

Re: ZKClient

[Mahadev Konar <ma...@yahoo-inc.com>]

Hi Adam,
  I don't think zk is very very hard to get right. There are exmaples in
src/recipes which implements locks/queues/others. There is ZOOKEEPER-22 to
make it even more easier for application to use.

Regarding re registration of watches, you can deifnitely write code and
submit is as a part of well documented contrib module which lays out the
assumptions/design of it. It could very well be useful for others. Its just
that folks havent had much time to focus on these areas as yet.

Thanks
mahadev


On 5/4/10 2:58 PM, "Adam Rosien" <ad...@rosien.net> wrote:

> I use zkclient in my work at kaChing and I have mixed feelings about
> it. On one hand it makes "easy things easy" which is great, but on the
> other hand I very few ideas what assumptions it makes "under the
> hood". I also dislike some of the design choices such as unchecked
> exceptions, but that's neither here nor there. It would take some
> extensive documentation work by the authors to really enumerate the
> model and assumptions, but the project doesn't seem to be active
> (either from it being adequate for its current users or just
> inactive). I'm not sure I could derive the assumptions myself.
> 
> I'm a bit frustrated that zk is "very, very hard to really get right".
> At a project level, can't we create structures to avoid most of these
> errors? Can there be a "standard model" with detailed assumptions and
> implementations of all the recipes? How can we start this? Is there
> something that makes this too hard?
> 
> I feel like a recipe page is a big fail; wouldn't an example app that
> uses locks and barriers be that much more compelling?
> 
> For the common FAQ items like "you need to re-register the watch",
> can't we just create code that implements this pattern? My goal is to
> live up to the motto: a good API is impossible to use incorrectly.
> 
> .. Adam
> 
> On Tue, May 4, 2010 at 2:21 PM, Ted Dunning <te...@gmail.com> wrote:
>> In general, writing this sort of layer on top of ZK is very, very hard to
>> get really right for general use.  In a simple use-case, you can probably
>> nail it but distributed systems are a Zoo, to coin a phrase.  The problem is
>> that you are fundamentally changing the metaphors in use so assumptions can
>> come unglued or be introduced pretty easily.
>> 
>> One example of this is the fact that ZK watches *don't* fire for every
>> change but when you write listener oriented code, you kind of expect that
>> they will.  That makes it really, really easy to introduce that assumption
>> in the heads of the programmer using the event listener library on top of
>> ZK.  Another example is how the atomic get content/set watch call works in
>> ZK is easy to violate in an event driven architecture because the thread
>> that watches ZK probably resets the watch.  If you assume that the listener
>> will read the data, then you have introduced a timing mismatch between the
>> read of the data and the resetting of the watch.  That might be OK or it
>> might not be.  The point is that these changes are subtle and tricky to get
>> exactly right.
>> 
>> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
>> jonathan.holloway@gmail.com> wrote:
>> 
>>> Is there any reason why this isn't part of the Zookeeper trunk already?
>>> 
>> 

Re: ZKClient

[Adam Rosien <ad...@rosien.net>]

I use zkclient in my work at kaChing and I have mixed feelings about
it. On one hand it makes "easy things easy" which is great, but on the
other hand I very few ideas what assumptions it makes "under the
hood". I also dislike some of the design choices such as unchecked
exceptions, but that's neither here nor there. It would take some
extensive documentation work by the authors to really enumerate the
model and assumptions, but the project doesn't seem to be active
(either from it being adequate for its current users or just
inactive). I'm not sure I could derive the assumptions myself.

I'm a bit frustrated that zk is "very, very hard to really get right".
At a project level, can't we create structures to avoid most of these
errors? Can there be a "standard model" with detailed assumptions and
implementations of all the recipes? How can we start this? Is there
something that makes this too hard?

I feel like a recipe page is a big fail; wouldn't an example app that
uses locks and barriers be that much more compelling?

For the common FAQ items like "you need to re-register the watch",
can't we just create code that implements this pattern? My goal is to
live up to the motto: a good API is impossible to use incorrectly.

.. Adam

On Tue, May 4, 2010 at 2:21 PM, Ted Dunning <te...@gmail.com> wrote:
> In general, writing this sort of layer on top of ZK is very, very hard to
> get really right for general use.  In a simple use-case, you can probably
> nail it but distributed systems are a Zoo, to coin a phrase.  The problem is
> that you are fundamentally changing the metaphors in use so assumptions can
> come unglued or be introduced pretty easily.
>
> One example of this is the fact that ZK watches *don't* fire for every
> change but when you write listener oriented code, you kind of expect that
> they will.  That makes it really, really easy to introduce that assumption
> in the heads of the programmer using the event listener library on top of
> ZK.  Another example is how the atomic get content/set watch call works in
> ZK is easy to violate in an event driven architecture because the thread
> that watches ZK probably resets the watch.  If you assume that the listener
> will read the data, then you have introduced a timing mismatch between the
> read of the data and the resetting of the watch.  That might be OK or it
> might not be.  The point is that these changes are subtle and tricky to get
> exactly right.
>
> On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
> jonathan.holloway@gmail.com> wrote:
>
>> Is there any reason why this isn't part of the Zookeeper trunk already?
>>
>

Re: ZKClient

[Ted Dunning <te...@gmail.com>]

In general, writing this sort of layer on top of ZK is very, very hard to
get really right for general use.  In a simple use-case, you can probably
nail it but distributed systems are a Zoo, to coin a phrase.  The problem is
that you are fundamentally changing the metaphors in use so assumptions can
come unglued or be introduced pretty easily.

One example of this is the fact that ZK watches *don't* fire for every
change but when you write listener oriented code, you kind of expect that
they will.  That makes it really, really easy to introduce that assumption
in the heads of the programmer using the event listener library on top of
ZK.  Another example is how the atomic get content/set watch call works in
ZK is easy to violate in an event driven architecture because the thread
that watches ZK probably resets the watch.  If you assume that the listener
will read the data, then you have introduced a timing mismatch between the
read of the data and the resetting of the watch.  That might be OK or it
might not be.  The point is that these changes are subtle and tricky to get
exactly right.

On Tue, May 4, 2010 at 1:48 PM, Jonathan Holloway <
jonathan.holloway@gmail.com> wrote:

> Is there any reason why this isn't part of the Zookeeper trunk already?
>

Re: ZKClient

[Patrick Hunt <ph...@apache.org>]

Take a look at this thread for some background.
http://www.mail-archive.com/zookeeper-user@hadoop.apache.org/msg00917.html

There were some concerns at the time, not sure if they have been 
addressed since (It has been a while since that discussion).

Patrick

On 05/04/2010 01:48 PM, Jonathan Holloway wrote:
> It looks good, having written a client already myself, I'd rather use this
> than have to roll
> my own each time.
>
> Is there any reason why this isn't part of the Zookeeper trunk already?
> It would make working with Zookeeper a bit easier (at least from my
> perspective)...
>
> Jon.
>
>
> On 4 May 2010 12:57, Ted Dunning<te...@gmail.com>  wrote:
>
>> This is used as part of katta where it gets a fair bit of exercise at low
>> update rates with small data.  It is used for managing the state of the
>> search cluster.
>>
>> I don't think it has had much external review or use for purposes apart
>> from
>> katta.  Katta generally has pretty decent code, though.
>>
>> On Tue, May 4, 2010 at 12:39 PM, Jonathan Holloway<
>> jonathan.holloway@gmail.com>  wrote:
>>
>>> I came across this project on Github
>>>
>>> http://github.com/sgroschupf/zkclient
>>>
>>> for working with the Zookeeper API.  Has anybody used it in the past?  Is
>>> it
>>> a better way of interacting with
>>> a Zookeeper cluster?
>>>
>>> Many thanks,
>>> Jon.
>>>
>>
>

Re: ZKClient

[Jonathan Holloway <jo...@gmail.com>]

It looks good, having written a client already myself, I'd rather use this
than have to roll
my own each time.

Is there any reason why this isn't part of the Zookeeper trunk already?
It would make working with Zookeeper a bit easier (at least from my
perspective)...

Jon.


On 4 May 2010 12:57, Ted Dunning <te...@gmail.com> wrote:

> This is used as part of katta where it gets a fair bit of exercise at low
> update rates with small data.  It is used for managing the state of the
> search cluster.
>
> I don't think it has had much external review or use for purposes apart
> from
> katta.  Katta generally has pretty decent code, though.
>
> On Tue, May 4, 2010 at 12:39 PM, Jonathan Holloway <
> jonathan.holloway@gmail.com> wrote:
>
> > I came across this project on Github
> >
> > http://github.com/sgroschupf/zkclient
> >
> > for working with the Zookeeper API.  Has anybody used it in the past?  Is
> > it
> > a better way of interacting with
> > a Zookeeper cluster?
> >
> > Many thanks,
> > Jon.
> >
>

Re: ZKClient

[Ted Dunning <te...@gmail.com>]

This is used as part of katta where it gets a fair bit of exercise at low
update rates with small data.  It is used for managing the state of the
search cluster.

I don't think it has had much external review or use for purposes apart from
katta.  Katta generally has pretty decent code, though.

On Tue, May 4, 2010 at 12:39 PM, Jonathan Holloway <
jonathan.holloway@gmail.com> wrote:

> I came across this project on Github
>
> http://github.com/sgroschupf/zkclient
>
> for working with the Zookeeper API.  Has anybody used it in the past?  Is
> it
> a better way of interacting with
> a Zookeeper cluster?
>
> Many thanks,
> Jon.
>