You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Alessio Pagliari <pa...@i3s.unice.fr> on 2018/04/20 14:16:28 UTC

Storm benchmarks: UI stat updates

Hi everybody,

I’m doing some benchmarking to check the behaviour of Storm over time, so I’m just retrieving each 10 seconds the total amount of tuples emitted per executor, via the UI APIs, and from that I compute the last 10 seconds value. I noticed that pretty regularly I obtain a peak in the measurements, for example: all the velues are around 500k tuples/10s, but sometime I get a value of ~1M tuples/10s. 

I wanted to know how the UI stats are updated with which frequency, and if it’s normal that I get this outliers in the measurements. Does someone have an explanation? Or a link where I can deepen this aspect.

Thank you,

----------
Alessio Pagliari
Scale/Signet Team, PhD Student
Université Côte d’Azur, CNRS, I3S
Website: http://www.i3s.unice.fr/~pagliari/ <http://www.i3s.unice.fr/~pagliari/>

Re: Storm benchmarks: UI stat updates

Posted by Roshan Naik <ro...@yahoo.com>.
It seems most metrics related settings are set to 60 sec by default.  

 

executor.metrics.frequency.secs : 60   

storm.cluster.metrics.consumer.publish.interval.secs: 60 

topology.builtin.metrics.bucket.size.secs: 60

 

-roshan

 

 

From: Alessio Pagliari <pa...@i3s.unice.fr>
Reply-To: <us...@storm.apache.org>
Date: Friday, April 20, 2018 at 7:22 AM
To: <us...@storm.apache.org>
Subject: Storm benchmarks: UI stat updates

 

Hi everybody,

 

I’m doing some benchmarking to check the behaviour of Storm over time, so I’m just retrieving each 10 seconds the total amount of tuples emitted per executor, via the UI APIs, and from that I compute the last 10 seconds value. I noticed that pretty regularly I obtain a peak in the measurements, for example: all the velues are around 500k tuples/10s, but sometime I get a value of ~1M tuples/10s. 

 

I wanted to know how the UI stats are updated with which frequency, and if it’s normal that I get this outliers in the measurements. Does someone have an explanation? Or a link where I can deepen this aspect.

 

Thank you,

 

----------
Alessio Pagliari

Scale/Signet Team, PhD Student

Université Côte d’Azur, CNRS, I3S

Website: http://www.i3s.unice.fr/~pagliari/