You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Tetsuya MATSUDA <te...@gmail.com> on 2015/01/10 00:48:08 UTC
How to count correct acked, failed, emitted etc?
Hi all,
I want to know correct number of acked, failed, emitted etc of my topology.
In StormUI, they are not correct because of rounding off.
Is there any way of counting these params?
Thanks,
--
Tetsuya MATSUDA
Re: How to count correct acked, failed, emitted etc?
Posted by Tetsuya MATSUDA <te...@gmail.com>.
Hi Jens,
Thanks for your replying and good idea.
I will try to count at spouts(in ack or fail block) :)
Regards,
Tetsuya
On Tue, Jan 13, 2015 at 9:13 PM, Jens-U. Mozdzen <jm...@nde.ag> wrote:
> Hi Tetsuya,
>
> Zitat von Tetsuya MATSUDA <te...@gmail.com>:
>
> I agree the values are multiples of 20 and they're not row count.
>> So if possible, I want to get raw count of them.
>>
>> Now, I have only one idea.
>> I count all parameters myself like below,
>> --------------------
>> begin
>> handling tuple
>> dbclient.incr("ack_count")
>> ack(tup)
>> rescue => err
>> dbclient.incr("fail_count")
>> log.error(tup.id.to_s + ":" + err.to_s)
>> fail(tup)
>> end
>> --------------------
>>
>> Do you have other better ideas?
>>
>
> how about adding the counters to your spouts? These do the emits and are
> notified of acks and fails, so you can do the count "at the source" and
> per-stream.
>
> Regards,
> Jens
>
>
--
松田 徹也 -Tetsuya MATSUDA-
E-mail : tetsuyam.ggl@gmail.com <ko...@gmail.com>
Re: How to count correct acked, failed, emitted etc?
Posted by "Jens-U. Mozdzen" <jm...@nde.ag>.
Hi Tetsuya,
Zitat von Tetsuya MATSUDA <te...@gmail.com>:
> I agree the values are multiples of 20 and they're not row count.
> So if possible, I want to get raw count of them.
>
> Now, I have only one idea.
> I count all parameters myself like below,
> --------------------
> begin
> handling tuple
> dbclient.incr("ack_count")
> ack(tup)
> rescue => err
> dbclient.incr("fail_count")
> log.error(tup.id.to_s + ":" + err.to_s)
> fail(tup)
> end
> --------------------
>
> Do you have other better ideas?
how about adding the counters to your spouts? These do the emits and
are notified of acks and fails, so you can do the count "at the
source" and per-stream.
Regards,
Jens
Re: How to count correct acked, failed, emitted etc?
Posted by Tetsuya MATSUDA <te...@gmail.com>.
I agree the values are multiples of 20 and they're not row count.
So if possible, I want to get raw count of them.
Now, I have only one idea.
I count all parameters myself like below,
--------------------
begin
handling tuple
dbclient.incr("ack_count")
ack(tup)
rescue => err
dbclient.incr("fail_count")
log.error(tup.id.to_s + ":" + err.to_s)
fail(tup)
end
--------------------
Do you have other better ideas?
On Sun, Jan 11, 2015 at 11:53 PM, Nathan Leung <nc...@gmail.com> wrote:
> UI represents values as multiples of 20, and as I understand it's not a
> raw count but a sample (I could be wrong about the second part).
>
> On Sun, Jan 11, 2015 at 9:34 AM, Tetsuya MATSUDA <te...@gmail.com>
> wrote:
>
>> Thank you for replying.
>>
>> There is not written about rounding off in docs, but the result of my
>> experiment is rounded off.
>> I will read docs you taught me and source code.
>>
>> Thanks,
>>
>> On Sat, Jan 10, 2015 at 9:35 AM, UmaraDissa1 . <um...@gmail.com>
>> wrote:
>>
>>> I don't know of any rounding off the UI does, is this in the docs?
>>>
>>> It might be helpful for you to look at creating your own metric.
>>>
>>> Official documentation -
>>> http://storm.apache.org/documentation/Metrics.html
>>>
>>> Walkthrough - https://www.endgame.com/blog/storm-metrics-how-to.html
>>>
>>> On Sat, Jan 10, 2015 at 10:48 AM, Tetsuya MATSUDA <
>>> tetsuyam.ggl@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I want to know correct number of acked, failed, emitted etc of my
>>>> topology.
>>>> In StormUI, they are not correct because of rounding off.
>>>>
>>>> Is there any way of counting these params?
>>>>
>>>> Thanks,
>>>> --
>>>> Tetsuya MATSUDA
>>>>
>>>>
>>>
>>
>>
>> --
>> 松田 徹也 -Tetsuya MATSUDA-
>> E-mail : tetsuyam.ggl@gmail.com <ko...@gmail.com>
>>
>
>
--
松田 徹也 -Tetsuya MATSUDA-
E-mail : tetsuyam.ggl@gmail.com <ko...@gmail.com>
Re: How to count correct acked, failed, emitted etc?
Posted by Nathan Leung <nc...@gmail.com>.
UI represents values as multiples of 20, and as I understand it's not a raw
count but a sample (I could be wrong about the second part).
On Sun, Jan 11, 2015 at 9:34 AM, Tetsuya MATSUDA <te...@gmail.com>
wrote:
> Thank you for replying.
>
> There is not written about rounding off in docs, but the result of my
> experiment is rounded off.
> I will read docs you taught me and source code.
>
> Thanks,
>
> On Sat, Jan 10, 2015 at 9:35 AM, UmaraDissa1 . <um...@gmail.com>
> wrote:
>
>> I don't know of any rounding off the UI does, is this in the docs?
>>
>> It might be helpful for you to look at creating your own metric.
>>
>> Official documentation -
>> http://storm.apache.org/documentation/Metrics.html
>>
>> Walkthrough - https://www.endgame.com/blog/storm-metrics-how-to.html
>>
>> On Sat, Jan 10, 2015 at 10:48 AM, Tetsuya MATSUDA <tetsuyam.ggl@gmail.com
>> > wrote:
>>
>>> Hi all,
>>>
>>> I want to know correct number of acked, failed, emitted etc of my
>>> topology.
>>> In StormUI, they are not correct because of rounding off.
>>>
>>> Is there any way of counting these params?
>>>
>>> Thanks,
>>> --
>>> Tetsuya MATSUDA
>>>
>>>
>>
>
>
> --
> 松田 徹也 -Tetsuya MATSUDA-
> E-mail : tetsuyam.ggl@gmail.com <ko...@gmail.com>
>
Re: How to count correct acked, failed, emitted etc?
Posted by Tetsuya MATSUDA <te...@gmail.com>.
Thank you for replying.
There is not written about rounding off in docs, but the result of my
experiment is rounded off.
I will read docs you taught me and source code.
Thanks,
On Sat, Jan 10, 2015 at 9:35 AM, UmaraDissa1 . <um...@gmail.com> wrote:
> I don't know of any rounding off the UI does, is this in the docs?
>
> It might be helpful for you to look at creating your own metric.
>
> Official documentation -
> http://storm.apache.org/documentation/Metrics.html
>
> Walkthrough - https://www.endgame.com/blog/storm-metrics-how-to.html
>
> On Sat, Jan 10, 2015 at 10:48 AM, Tetsuya MATSUDA <te...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I want to know correct number of acked, failed, emitted etc of my
>> topology.
>> In StormUI, they are not correct because of rounding off.
>>
>> Is there any way of counting these params?
>>
>> Thanks,
>> --
>> Tetsuya MATSUDA
>>
>>
>
--
松田 徹也 -Tetsuya MATSUDA-
E-mail : tetsuyam.ggl@gmail.com <ko...@gmail.com>
Re: How to count correct acked, failed, emitted etc?
Posted by "UmaraDissa1 ." <um...@gmail.com>.
I don't know of any rounding off the UI does, is this in the docs?
It might be helpful for you to look at creating your own metric.
Official documentation - http://storm.apache.org/documentation/Metrics.html
Walkthrough - https://www.endgame.com/blog/storm-metrics-how-to.html
On Sat, Jan 10, 2015 at 10:48 AM, Tetsuya MATSUDA <te...@gmail.com>
wrote:
> Hi all,
>
> I want to know correct number of acked, failed, emitted etc of my topology.
> In StormUI, they are not correct because of rounding off.
>
> Is there any way of counting these params?
>
> Thanks,
> --
> Tetsuya MATSUDA
>
>