You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Xueming Li <ja...@gmail.com> on 2014/06/03 05:58:47 UTC

Custom metrics using IMetrics interface

Hi all,

I am working on a project to build an order processing pipeline on top of
Storm. In order to measure performance, for every spout/bolt and every
order processed by them, one requirement is to generate custom metrics in
the form <order id, order entry timestamp in milisec, order exit timestamp
in milisec> and then use a metrics consumer to collect and further process
those metrics. I will greatly appreciate it if any one can share some ideas
as to how to implement this requirement. My question is, getValueAndReset
in IMetrics interface is a callback method. Looks to me that in order
to collect the metrics using IMetrics interface, I will have to use a
memory queue to temporarily store metrics generated when orders are
processed in any spout and bolt and wait for getValueAndReset to be called.
Any better idea?

Thanks,
James

Re: Custom metrics using IMetrics interface

Posted by Bobby Evans <ev...@yahoo-inc.com>.
IMetric and the metrics parts in storm are built on top of storm itself.  If you need something that sends out metrics more regularly then Imetric does, just send them as regular tuples to a bolt you setup to collect them.

- Bobby

From: Xueming Li <ja...@gmail.com>>
Reply-To: "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Date: Monday, June 2, 2014 at 10:58 PM
To: "dev@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <de...@storm.incubator.apache.org>>, "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Subject: Custom metrics using IMetrics interface

Hi all,

I am working on a project to build an order processing pipeline on top of Storm. In order to measure performance, for every spout/bolt and every order processed by them, one requirement is to generate custom metrics in the form <order id, order entry timestamp in milisec, order exit timestamp in milisec> and then use a metrics consumer to collect and further process those metrics. I will greatly appreciate it if any one can share some ideas as to how to implement this requirement. My question is, getValueAndReset in IMetrics interface is a callback method. Looks to me that in order to collect the metrics using IMetrics interface, I will have to use a memory queue to temporarily store metrics generated when orders are processed in any spout and bolt and wait for getValueAndReset to be called. Any better idea?

Thanks,
James

Re: Custom metrics using IMetrics interface

Posted by Bobby Evans <ev...@yahoo-inc.com.INVALID>.
IMetric and the metrics parts in storm are built on top of storm itself.  If you need something that sends out metrics more regularly then Imetric does, just send them as regular tuples to a bolt you setup to collect them.

- Bobby

From: Xueming Li <ja...@gmail.com>>
Reply-To: "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Date: Monday, June 2, 2014 at 10:58 PM
To: "dev@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <de...@storm.incubator.apache.org>>, "user@storm.incubator.apache.org<ma...@storm.incubator.apache.org>" <us...@storm.incubator.apache.org>>
Subject: Custom metrics using IMetrics interface

Hi all,

I am working on a project to build an order processing pipeline on top of Storm. In order to measure performance, for every spout/bolt and every order processed by them, one requirement is to generate custom metrics in the form <order id, order entry timestamp in milisec, order exit timestamp in milisec> and then use a metrics consumer to collect and further process those metrics. I will greatly appreciate it if any one can share some ideas as to how to implement this requirement. My question is, getValueAndReset in IMetrics interface is a callback method. Looks to me that in order to collect the metrics using IMetrics interface, I will have to use a memory queue to temporarily store metrics generated when orders are processed in any spout and bolt and wait for getValueAndReset to be called. Any better idea?

Thanks,
James