Metrics Period

[Dave Webb <Da...@gmx.de>]

Hi,

I'm experimenting with Storm Metrics and stumbled upon some strange behavior.
For some testing, I registered a CountMetric with a very short bucket time of 1 seconds:

        context.registerMetric("count", countMetric, 1);

First, I ran the topology with *no* additional "OutputBolt", so all tuples are eventually discarded.
With this setup, my MetricsConsumer receives the Metric very regularly and the measured "count" was high.

...
2016-10-29 15:26:51.366 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:26:52.568 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:26:53.644 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:26:54.803 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
...
-> very regularly every 1 sec

Then, I added an additional "OutputBolt" which stores the final tuples in a database. Obviously, this causes
the measured "count" to significantly decrease, which I expected.
However, this setup also causes my MetricsConsumer to receive Updates *much* less regularly:

...
2016-10-29 15:39:52.555 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:38:29.732 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:39:13.162 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:39:45.867 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
...
-> not regularly: only every 30-45 seconds

I see that the load on the topology is high and it seems to "fill up" all queues, but it doesn't seem right
that the Metric consumption gets this bad so quickly. The machine itself has plenty of resources left to handle
proper Metric reporting.

Are there better way to receive Metrics with a higher frequency (like every 1-5 seconds)?