Are metrics always sent at the time bucket interval?

[Jake Dodd <ja...@ontopic.io>]

Hi all,

I have a bolt that registers a MultiCountMetric, with the time bucket set to 10 seconds. The metrics aren’t being sent until the bolt’s stream experiences a lull. For example, if I emit 10 tuples into the stream, and the bolt takes 8 seconds to process them, then the metrics will be sent every 10 seconds as expected. But if I emit 50 tuples, and the bolt takes 40 seconds to process them, there is a 40 second gap in the metrics. If there are no more tuples emitted into the stream after this, the bolt resumes sending metrics at 10-second intervals.

For reference, I’m using Storm 0.93. The bolt isn’t doing anything weird—just a straightforward execute method with an emit and ack at the end. Execute time is typically ~1 second. The only potential quirkiness is the fact that it establishes an HTTP connection, which times out after a max of 5 seconds.

Is this a bug, known issue, or intended behavior?

Cheers

Jake 

Re: Are metrics always sent at the time bucket interval?

[Nathan Marz <na...@nathanmarz.com>]

Metrics operates on the same thread as the bolt, so this is the expected
behavior. We did it this way so that metrics would be very fast – updating,
reading, and emitting metrics require no locking or synchronization
whatsoever.

On Mon, Oct 27, 2014 at 11:42 AM, Jake Dodd <ja...@ontopic.io> wrote:

> Hi all,
>
> I have a bolt that registers a MultiCountMetric, with the time bucket set
> to 10 seconds. The metrics aren’t being sent until the bolt’s stream
> experiences a lull. For example, if I emit 10 tuples into the stream, and
> the bolt takes 8 seconds to process them, then the metrics will be sent
> every 10 seconds as expected. But if I emit 50 tuples, and the bolt takes
> 40 seconds to process them, there is a 40 second gap in the metrics. If
> there are no more tuples emitted into the stream after this, the bolt
> resumes sending metrics at 10-second intervals.
>
> For reference, I’m using Storm 0.93. The bolt isn’t doing anything
> weird—just a straightforward execute method with an emit and ack at the
> end. Execute time is typically ~1 second. The only potential quirkiness is
> the fact that it establishes an HTTP connection, which times out after a
> max of 5 seconds.
>
> Is this a bug, known issue, or intended behavior?
>
> Cheers
>
> Jake




-- 
Twitter: @nathanmarz
http://nathanmarz.com