[Need help] [How to benchmark storm topologies]

[Abhishek Bhattacharjee <ab...@gmail.com>]

I wish to benchmark two topologies for storm, one is stateless and doesn't
store any intermediate state and tuples are replayed and re-processed when
failure occurs. Whereas the other is stateful and intermediate state is
saved.

I want to see the performance in both the cases with equal amount of
failures.
How should I go about it ?
Any help is appreciated.

Thanks,

*Abhishek Bhattacharjee*
*Pune Institute of Computer Technology*

Re: [Need help] [How to benchmark storm topologies]

[Danijel Schiavuzzi <da...@schiavuzzi.com>]

You have to use the Storm Metrics API, and implement a Storm Metrics
consumer to consume those metrics. For Statsd and Graphite, have a look at
the storm-metrics-statsd project on GitHub, which implements a metrics
consumer that forwards Storm's metrics to Statsd.

On Sunday, February 16, 2014, Abhishek Bhattacharjee <
abhishek.bhattacharjee11@gmail.com> wrote:

> Mark could you please elaborate how you are using statsD and Graphite with
> storm.
> I understand that statsD sends data to Graphite. But how are you using
> statsD for storm ?
> Could you give me any sample code or any links so that I can understand
> how to work with it.
>
> Thanks
>
>
> On Sun, Feb 16, 2014 at 3:40 AM, Abhishek Bhattacharjee <
> abhishek.bhattacharjee11@gmail.com<javascript:_e(%7B%7D,'cvml','abhishek.bhattacharjee11@gmail.com');>
> > wrote:
>
>> Thanks for the reply :)
>>
>>
>> On Sat, Feb 15, 2014 at 1:17 PM, Mark Hu <markhu101@gmail.com<javascript:_e(%7B%7D,'cvml','markhu101@gmail.com');>
>> > wrote:
>>
>>> We instrument our spout and bolts using statsd and graphite due to its
>>> ease of use and flexibility. Its UDP and won't add additional risk or
>>> vulnerabilities to your topologies.
>>>  On Feb 15, 2014 1:05 AM, "Abhishek Bhattacharjee" <
>>> abhishek.bhattacharjee11@gmail.com<javascript:_e(%7B%7D,'cvml','abhishek.bhattacharjee11@gmail.com');>>
>>> wrote:
>>>
>>>> I wish to benchmark two topologies for storm, one is stateless and
>>>> doesn't store any intermediate state and tuples are replayed and
>>>> re-processed when failure occurs. Whereas the other is stateful and
>>>> intermediate state is saved.
>>>>
>>>> I want to see the performance in both the cases with equal amount of
>>>> failures.
>>>> How should I go about it ?
>>>> Any help is appreciated.
>>>>
>>>> Thanks,
>>>>
>>>> *Abhishek Bhattacharjee*
>>>> *Pune Institute of Computer Technology*
>>>>
>>>
>>
>>
>> --
>> *Abhishek Bhattacharjee*
>> *Pune Institute of Computer Technology*
>>
>
>
>
> --
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>


-- 
Danijel Schiavuzzi

E: danijel@schiavuzzi.com
W: www.schiavuzzi.com
T: +385989035562
Skype: danijels7

Re: [Need help] [How to benchmark storm topologies]

[Abhishek Bhattacharjee <ab...@gmail.com>]

Mark could you please elaborate how you are using statsD and Graphite with
storm.
I understand that statsD sends data to Graphite. But how are you using
statsD for storm ?
Could you give me any sample code or any links so that I can understand how
to work with it.

Thanks


On Sun, Feb 16, 2014 at 3:40 AM, Abhishek Bhattacharjee <
abhishek.bhattacharjee11@gmail.com> wrote:

> Thanks for the reply :)
>
>
> On Sat, Feb 15, 2014 at 1:17 PM, Mark Hu <ma...@gmail.com> wrote:
>
>> We instrument our spout and bolts using statsd and graphite due to its
>> ease of use and flexibility. Its UDP and won't add additional risk or
>> vulnerabilities to your topologies.
>>  On Feb 15, 2014 1:05 AM, "Abhishek Bhattacharjee" <
>> abhishek.bhattacharjee11@gmail.com> wrote:
>>
>>> I wish to benchmark two topologies for storm, one is stateless and
>>> doesn't store any intermediate state and tuples are replayed and
>>> re-processed when failure occurs. Whereas the other is stateful and
>>> intermediate state is saved.
>>>
>>> I want to see the performance in both the cases with equal amount of
>>> failures.
>>> How should I go about it ?
>>> Any help is appreciated.
>>>
>>> Thanks,
>>>
>>> *Abhishek Bhattacharjee*
>>> *Pune Institute of Computer Technology*
>>>
>>
>
>
> --
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>



-- 
*Abhishek Bhattacharjee*
*Pune Institute of Computer Technology*

Re: [Need help] [How to benchmark storm topologies]

[Abhishek Bhattacharjee <ab...@gmail.com>]

Thanks for the reply :)


On Sat, Feb 15, 2014 at 1:17 PM, Mark Hu <ma...@gmail.com> wrote:

> We instrument our spout and bolts using statsd and graphite due to its
> ease of use and flexibility. Its UDP and won't add additional risk or
> vulnerabilities to your topologies.
> On Feb 15, 2014 1:05 AM, "Abhishek Bhattacharjee" <
> abhishek.bhattacharjee11@gmail.com> wrote:
>
>> I wish to benchmark two topologies for storm, one is stateless and
>> doesn't store any intermediate state and tuples are replayed and
>> re-processed when failure occurs. Whereas the other is stateful and
>> intermediate state is saved.
>>
>> I want to see the performance in both the cases with equal amount of
>> failures.
>> How should I go about it ?
>> Any help is appreciated.
>>
>> Thanks,
>>
>> *Abhishek Bhattacharjee*
>> *Pune Institute of Computer Technology*
>>
>


-- 
*Abhishek Bhattacharjee*
*Pune Institute of Computer Technology*

Re: [Need help] [How to benchmark storm topologies]

[Mark Hu <ma...@gmail.com>]

We instrument our spout and bolts using statsd and graphite due to its ease
of use and flexibility. Its UDP and won't add additional risk or
vulnerabilities to your topologies.
On Feb 15, 2014 1:05 AM, "Abhishek Bhattacharjee" <
abhishek.bhattacharjee11@gmail.com> wrote:

> I wish to benchmark two topologies for storm, one is stateless and doesn't
> store any intermediate state and tuples are replayed and re-processed when
> failure occurs. Whereas the other is stateful and intermediate state is
> saved.
>
> I want to see the performance in both the cases with equal amount of
> failures.
> How should I go about it ?
> Any help is appreciated.
>
> Thanks,
>
> *Abhishek Bhattacharjee*
> *Pune Institute of Computer Technology*
>