How to check the time a tuple spends waiting in an incoming Queue until it gets processed

["Nick R. Katsipoulakis" <ni...@gmail.com>]

Hello all,

I have been trying to calculate, as precisely as possible, how much time a
tuple spends waiting in the input queue of a Bolt until it is provided in
the BaseRichBolt.execute() function.

The way I try to get this wait time (latency) is by adding a timestamp
attribute in the tuple, the time it gets produced (I call Java's
System.currentTimeInMillis()). When it gets picked up from a downstream
Bolt (call of execute() function), I retrieve the timestamp attribute and
calculate the difference in milliseconds from the current timestamp.
However, due to clock skewness in my local cluster (I guess that this is
the cause), I usually retrieve difference values that do not make sense
(sometimes even negative wait times). I have previously tried NTP but I
still do not get values that make sense.

When I check the Storm UI reported metrics, I see that there two latency
metrics: Process latency and one more. Is there any way I can retrieve
those from the java code inside a Bolt's execute method? Is there a better
way to get a more precise latency metric from inside the execute() function
of a Bolt?

Thanks,
Nick

-- 
Nikolaos Romanos Katsipoulakis,
University of Pittsburgh, PhD candidate