[jira] [Commented] (STORM-1007) Add more metrics to DisruptorQueue

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728944#comment-14728944 ] 

ASF GitHub Bot commented on STORM-1007:
---------------------------------------

GitHub user wangli1426 opened a pull request:

    https://github.com/apache/storm/pull/716

    [STORM-1007] Added tuple arrival rate and tuple sojourn time to QueueMetrics

    I added two important metrics, namely *arrival time* and *sojourn time*, to QueueMetrics. 
    
    Modification Summary:
    1. Created a new RateTracker utility, which could report instantaneous rate upon certain event;
    2. Added test cases for RateTracker;
    3. Added arrival rate and sojourn time metrics to QueueMetrics; 
    4. Use sampling in QueueMetrics to reduce the metric maintenance cost. 
    
    Such metrics are very useful to performance tuning, especially for application with strong constraint on processing latency such as thread detection and smart traffic. A high sojourn time, for example, indicates that a higher degree of parallelism should be applied to the current bolt in order to reduce processing delay. 
    
    Any comment or suggestion regarding to this pull request is welcomed. 
    
    Thanks.
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/wangli1426/storm queue-metrics

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/storm/pull/716.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #716
    
----
commit 3d0e79e56089023a0b731fc2f4f66f24cd3264e1
Author: wangli1426 <wa...@gmail.com>
Date:   2015-08-26T09:40:34Z

    Added arrival_rate metric to DisruptorQueue.

commit 9cd94bb865c527fb8f5c6728670f97cb9073e0f8
Author: Li Wang <wa...@gmail.com>
Date:   2015-09-03T07:57:15Z

    Added Tests for RateTracker

commit fbf85db2f152440fffebd432f3106ddfdf1a848d
Author: Li Wang <wa...@gmail.com>
Date:   2015-09-03T07:58:24Z

    Added sampling feature to QueueMetrics to reduce the costs.

commit d026c49c594fdcbbb2665c116b1446798400b0bf
Author: wangli1426 <wa...@gmail.com>
Date:   2015-09-03T11:51:51Z

    Added element sojourn time, a new metric, to QueueMetrics.

----


> Add more metrics to DisruptorQueue
> ----------------------------------
>
>                 Key: STORM-1007
>                 URL: https://issues.apache.org/jira/browse/STORM-1007
>             Project: Apache Storm
>          Issue Type: Bug
>            Reporter: Li Wang
>            Assignee: Li Wang
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> The metrics of the queues for each component are very helpful to reason about the performance issues of a topology.
> For instance, for the applications with strong time constraint (e.g., threat detection), if the elements in the input queue of a bolt have a long sojourn time, it indicates that the user should increase the parallelism of that bolt to reduce the processing delay.
> However, the metrics on the DisruptorQueue currently available are limited. More useful metrics, such as average sojourn time and average throughput are expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)