what is the maximum watches allowed per znode

[Nikhil <mn...@gmail.com>]

Hi,

I am wondering if there is any limit as to how many watches can be allowed
per znode in the zookeeper quorum and how is this determined?

Also, does having so many watches on lot of znodes affect the zookeeper
performance? I am looking at the watch count metric on zookeeper cluster
and see that that watch count has been steadily rising and so I am
wondering whether there are any performance attributions because of the
rising watch count.

thanks,
Nikhil

Re: what is the maximum watches allowed per znode

[kishore g <g....@gmail.com>]

If you care about performance with large number watches and clients, you
might want to look at this
https://issues.apache.org/jira/browse/ZOOKEEPER-1177.




On Wed, Feb 5, 2014 at 4:59 PM, Raúl Gutiérrez Segalés
<rg...@itevenworks.net>wrote:

> Hi,
>
> On 5 February 2014 15:42, Nikhil <mn...@gmail.com> wrote:
>
> > Hi,
> >
> > I am wondering if there is any limit as to how many watches can be
> allowed
> > per znode in the zookeeper quorum and how is this determined?
> >
>
> There is no limit other than the practical ones imposed by system resources
> (i.e.: the memory consumed by each watch, the CPU time spend on event
> delivery, etc).
>
> You can look at how a watch is set here:
>
>
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/FinalRequestProcessor.java#L328
>
> and here:
>
>
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/DataTree.java#L1291
>
> and finally (see the HashMaps) here:
>
>
> https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/WatchManager.java#L55
>
>
> >
> > Also, does having so many watches on lot of znodes affect the zookeeper
> > performance?
>
>
> Sure, because you need to pay for the allocation space in the HashMaps
> mentioned above, the CPU time of delivering events (i.e.: DataChanged) and
> (more importantly maybe?) on every re-connect you'll need to send your list
> of watches to have them re-registered. This can cause a lot of pressure
> during network events which trigger massive reconnect waves.
>
>
> -rgs
>

Re: what is the maximum watches allowed per znode

[Raúl Gutiérrez Segalés <rg...@itevenworks.net>]

Hi,

On 5 February 2014 15:42, Nikhil <mn...@gmail.com> wrote:

> Hi,
>
> I am wondering if there is any limit as to how many watches can be allowed
> per znode in the zookeeper quorum and how is this determined?
>

There is no limit other than the practical ones imposed by system resources
(i.e.: the memory consumed by each watch, the CPU time spend on event
delivery, etc).

You can look at how a watch is set here:

https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/FinalRequestProcessor.java#L328

and here:

https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/DataTree.java#L1291

and finally (see the HashMaps) here:

https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/server/WatchManager.java#L55


>
> Also, does having so many watches on lot of znodes affect the zookeeper
> performance?


Sure, because you need to pay for the allocation space in the HashMaps
mentioned above, the CPU time of delivering events (i.e.: DataChanged) and
(more importantly maybe?) on every re-connect you'll need to send your list
of watches to have them re-registered. This can cause a lot of pressure
during network events which trigger massive reconnect waves.


-rgs