[jira] [Commented] (STORM-2610) Spout throtteling metrics are unusable

["Robert Joseph Evans (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070627#comment-16070627 ] 

Robert Joseph Evans commented on STORM-2610:
--------------------------------------------

Actually the name is bad, so instead I am going to also rename them to be descriptive.

> Spout throtteling metrics are unusable
> --------------------------------------
>
>                 Key: STORM-2610
>                 URL: https://issues.apache.org/jira/browse/STORM-2610
>             Project: Apache Storm
>          Issue Type: New Feature
>          Components: storm-client, storm-core
>    Affects Versions: 1.0.0, 2.0.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>
> When helping someone debug an issue with backpressure I realized that the metrics we are collecting in the spout are mistakenly being multiplied by the rate, even though we are not sub-sampling them.  This results in the values being, by default, 20 times higher then they should be.  Thinking about how I would use the metrics to debug an issue also showed that some of them.  skipped-max-spout and skipped-throttle correspond to about 1 ms of sleep, but skipped-inactive corresponds to about 100 ms of sleep.  And the 1 ms sleep is configurable so it could be different from one topology to another, and even the code around it is pluggable, so it could be doing anything from not sleeping to sleeping a random amount of time.
> I think we just need to scrap what we have been doing and record how long we sleep for and use that as the metric instead.
> These metrics also don't appear to be documented anywhere so I am going to change what they mean and document them to actually be useful, and correct.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)