[jira] [Commented] (ACCUMULO-2757) millions of zookeeper watches

["Eric Newton (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ACCUMULO-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049281#comment-14049281 ] 

Eric Newton commented on ACCUMULO-2757:
---------------------------------------

On a moderate sized cluster (2 racks, 78 tservers), sitting idle with no clients there are 13804 watches.  Each client (the shell) doing nothing adds 91 more watches.  Clients probably don't even need to set up any.  Namespaces probably doubled the number of watches set.

I have started looking into jamming the config into a single node, and it's going to be a very big change.

Need to investigate the limits of how many watches a zookeeper cluster can manage.




> millions of zookeeper watches
> -----------------------------
>
>                 Key: ACCUMULO-2757
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2757
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: client, master, tserver
>         Environment: very large clusters
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>
> I recently learned how to get the number of watches in a zookeeper node.
> {noformat}
> $ echo wchp | nc localhost 2181
> {noformat}
> There are a surprising number of watches supporting table configuration.  A single tserver accumulo instance with one data table is using 872 watches.  Large clusters, even with many zookeepers, can have millions of watches.
> We may need to watch a single notification node, or serialize the table configs into a single node per table.  That would accomplish ACCUMULO-1568, too.



--
This message was sent by Atlassian JIRA
(v6.2#6252)