You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@storm.apache.org by danny0405 <gi...@git.apache.org> on 2017/09/25 07:23:37 UTC

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

Github user danny0405 commented on the issue:

    https://github.com/apache/storm/pull/2203
  
    @HeartSaVioR @revans2  I have read most of the tickets and issues of STORM-2153, there are many discussions about the metrics V2 and the now built-in metrics. But i didn't see any about the heartbeats part.
    
    Actually the heartbeats now write too much things[ window stats of tasks together] to zookeeper[ or pacemaker ], and we report it for worker granularity which actually can be pre-aggregated by machine/supervisor.
    
    With such heavy implementation, we have serious efficiency reduction when cluster grows to be large, like what i have described in JIRA task: https://issues.apache.org/jira/browse/STORM-2693.
    
    So i wanna promote the heartbeats part, which can be done individually apart the metrics V2 thing, this is what i will design the heartbeats to be:
    
    ![image](https://user-images.githubusercontent.com/7644508/30796521-fd6335fc-a204-11e7-95e5-1c3e08175b57.png)
    
    After the promotion, we do not need to read the heartbeats from zookeeper any more when scheduling which will cost much time. But opening Ui will still be slow[ not a big deal anyway ].


---