[jira] [Commented] (STORM-375) Smarter downloading of assignments by supervisors and workers

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14213939#comment-14213939 ] 

ASF GitHub Bot commented on STORM-375:
--------------------------------------

Github user xiaokang commented on the pull request:

    https://github.com/apache/storm/pull/167#issuecomment-63222238
  
    In our production storm cluster with 600 nodes, ZK read bandwidth had been reduced by 75% by applying this PR and simple fix for the "swap!" mistake I mentioned above.
    
    However the mistake of using "swap!" instead of "reset!" in (swap! (:assignment-versions supervisor) versions) is still in master branch. @knusbaum & @nathanmarz , any idea?


> Smarter downloading of assignments by supervisors and workers
> -------------------------------------------------------------
>
>                 Key: STORM-375
>                 URL: https://issues.apache.org/jira/browse/STORM-375
>             Project: Apache Storm
>          Issue Type: Improvement
>            Reporter: Kyle Nusbaum
>            Assignee: Kyle Nusbaum
>             Fix For: 0.9.3
>
>
> Periodically the supervisors and workers each download their assignments from zookeeper, and the traffic begins to saturate the network and hammer zookeeper as we scale. Having these daemons only grab the assignments when the zookeeper path's version number changes reduces this traffic considerably, along with STORM-376.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)