You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by researcher cs <pr...@gmail.com> on 2016/01/13 15:51:17 UTC
topology.stats.sample.rate
what is this properties used for topology.stats.sample.rate ?
Re: topology.stats.sample.rate
Posted by John Yost <ho...@gmail.com>.
The other config parameters will not affect the acking, which is the first
thing you are trying to figure out. I think you are actually only
processing tick tuples, so I recommend inserting logging statements in bolt
b-0 to confirm.
--John
On Sun, Jan 17, 2016 at 1:12 AM, researcher cs <pr...@gmail.com>
wrote:
> @john thanks I'll try that but I'm asking also about other properties , in
> what conditions I should change of default values , and is
> max.spout.pounding is related to tweets that I set it in the code to emit
> it ?
> I mean if I set to get 500 tweets then I should set max.spout.pounding=500
> Sorry for my lot questions but thanks for your helping .
>
>
> On Sunday, January 17, 2016, John Yost <ho...@gmail.com> wrote:
>
>> Hey, I recommend 1.0 for now and see if that solves the no ack issue. I
>> recommend max.spout.pending at 10K to hopefully ensure you avoid tuple
>> failures and then move up from there if you see back pressure. Back
>> pressure you'll see in the form of Bolt capacities way less than 0.5.
>>
>> --John
>>
>> On Sat, Jan 16, 2016 at 12:29 PM, researcher cs <
>> prog.researcher@gmail.com> wrote:
>>
>>> @john thanks for your time and help , i need to know is value 1.0 is the
>>> best value should i use or can i increase it ?
>>> max.spout.pounding = 50000 or 50K so what are other properties should i
>>> change from default value in storm.yaml like
>>> topology.acker.executors
>>> topology.executor.receive.buffer.size 1024
>>> topology.executor.send.buffer.size 1024
>>>
>>> i have one worker - one supervisor --
>>>
>>>
>>>
>>> On Sat, Jan 16, 2016 at 1:39 PM, John Yost <ho...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Argh, I apologize, just getting back to this. The storm default
>>>> topology.stats.sample.rate is 0.05, I set it to 1.0 for mine, and that's
>>>> when I started seeing numbers for executed/emitted/acked. I think the
>>>> sample rate needs to be higher for lower throughput installations like
>>>> mine, and maybe your's as well.
>>>>
>>>> Again, sorry for the delay--hope this works for you also!
>>>>
>>>> --John
>>>>
>>>>
>>>> On Wed, Jan 13, 2016 at 9:53 AM, Stephen Powis <sp...@salesforce.com>
>>>> wrote:
>>>>
>>>>>
>>>>> http://stackoverflow.com/questions/26276096/performace-impact-of-using-setstatssamplerate-topology-stats-sample-rate
>>>>>
>>>>> On Wed, Jan 13, 2016 at 9:51 PM, researcher cs <
>>>>> prog.researcher@gmail.com> wrote:
>>>>>
>>>>>> what is this properties used for topology.stats.sample.rate ?
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
Re: topology.stats.sample.rate
Posted by researcher cs <pr...@gmail.com>.
@john thanks I'll try that but I'm asking also about other properties , in
what conditions I should change of default values , and is
max.spout.pounding is related to tweets that I set it in the code to emit
it ?
I mean if I set to get 500 tweets then I should set max.spout.pounding=500
Sorry for my lot questions but thanks for your helping .
On Sunday, January 17, 2016, John Yost <ho...@gmail.com> wrote:
> Hey, I recommend 1.0 for now and see if that solves the no ack issue. I
> recommend max.spout.pending at 10K to hopefully ensure you avoid tuple
> failures and then move up from there if you see back pressure. Back
> pressure you'll see in the form of Bolt capacities way less than 0.5.
>
> --John
>
> On Sat, Jan 16, 2016 at 12:29 PM, researcher cs <prog.researcher@gmail.com
> <javascript:_e(%7B%7D,'cvml','prog.researcher@gmail.com');>> wrote:
>
>> @john thanks for your time and help , i need to know is value 1.0 is the
>> best value should i use or can i increase it ?
>> max.spout.pounding = 50000 or 50K so what are other properties should i
>> change from default value in storm.yaml like
>> topology.acker.executors
>> topology.executor.receive.buffer.size 1024
>> topology.executor.send.buffer.size 1024
>>
>> i have one worker - one supervisor --
>>
>>
>>
>> On Sat, Jan 16, 2016 at 1:39 PM, John Yost <hokiegeek2@gmail.com
>> <javascript:_e(%7B%7D,'cvml','hokiegeek2@gmail.com');>> wrote:
>>
>>> Hi,
>>>
>>> Argh, I apologize, just getting back to this. The storm default
>>> topology.stats.sample.rate is 0.05, I set it to 1.0 for mine, and that's
>>> when I started seeing numbers for executed/emitted/acked. I think the
>>> sample rate needs to be higher for lower throughput installations like
>>> mine, and maybe your's as well.
>>>
>>> Again, sorry for the delay--hope this works for you also!
>>>
>>> --John
>>>
>>>
>>> On Wed, Jan 13, 2016 at 9:53 AM, Stephen Powis <spowis@salesforce.com
>>> <javascript:_e(%7B%7D,'cvml','spowis@salesforce.com');>> wrote:
>>>
>>>>
>>>> http://stackoverflow.com/questions/26276096/performace-impact-of-using-setstatssamplerate-topology-stats-sample-rate
>>>>
>>>> On Wed, Jan 13, 2016 at 9:51 PM, researcher cs <
>>>> prog.researcher@gmail.com
>>>> <javascript:_e(%7B%7D,'cvml','prog.researcher@gmail.com');>> wrote:
>>>>
>>>>> what is this properties used for topology.stats.sample.rate ?
>>>>>
>>>>
>>>>
>>>
>>
>
Re: topology.stats.sample.rate
Posted by John Yost <ho...@gmail.com>.
Hey, I recommend 1.0 for now and see if that solves the no ack issue. I
recommend max.spout.pending at 10K to hopefully ensure you avoid tuple
failures and then move up from there if you see back pressure. Back
pressure you'll see in the form of Bolt capacities way less than 0.5.
--John
On Sat, Jan 16, 2016 at 12:29 PM, researcher cs <pr...@gmail.com>
wrote:
> @john thanks for your time and help , i need to know is value 1.0 is the
> best value should i use or can i increase it ?
> max.spout.pounding = 50000 or 50K so what are other properties should i
> change from default value in storm.yaml like
> topology.acker.executors
> topology.executor.receive.buffer.size 1024
> topology.executor.send.buffer.size 1024
>
> i have one worker - one supervisor --
>
>
>
> On Sat, Jan 16, 2016 at 1:39 PM, John Yost <ho...@gmail.com> wrote:
>
>> Hi,
>>
>> Argh, I apologize, just getting back to this. The storm default
>> topology.stats.sample.rate is 0.05, I set it to 1.0 for mine, and that's
>> when I started seeing numbers for executed/emitted/acked. I think the
>> sample rate needs to be higher for lower throughput installations like
>> mine, and maybe your's as well.
>>
>> Again, sorry for the delay--hope this works for you also!
>>
>> --John
>>
>>
>> On Wed, Jan 13, 2016 at 9:53 AM, Stephen Powis <sp...@salesforce.com>
>> wrote:
>>
>>>
>>> http://stackoverflow.com/questions/26276096/performace-impact-of-using-setstatssamplerate-topology-stats-sample-rate
>>>
>>> On Wed, Jan 13, 2016 at 9:51 PM, researcher cs <
>>> prog.researcher@gmail.com> wrote:
>>>
>>>> what is this properties used for topology.stats.sample.rate ?
>>>>
>>>
>>>
>>
>
Re: topology.stats.sample.rate
Posted by researcher cs <pr...@gmail.com>.
@john thanks for your time and help , i need to know is value 1.0 is the
best value should i use or can i increase it ?
max.spout.pounding = 50000 or 50K so what are other properties should i
change from default value in storm.yaml like
topology.acker.executors
topology.executor.receive.buffer.size 1024
topology.executor.send.buffer.size 1024
i have one worker - one supervisor --
On Sat, Jan 16, 2016 at 1:39 PM, John Yost <ho...@gmail.com> wrote:
> Hi,
>
> Argh, I apologize, just getting back to this. The storm default
> topology.stats.sample.rate is 0.05, I set it to 1.0 for mine, and that's
> when I started seeing numbers for executed/emitted/acked. I think the
> sample rate needs to be higher for lower throughput installations like
> mine, and maybe your's as well.
>
> Again, sorry for the delay--hope this works for you also!
>
> --John
>
>
> On Wed, Jan 13, 2016 at 9:53 AM, Stephen Powis <sp...@salesforce.com>
> wrote:
>
>>
>> http://stackoverflow.com/questions/26276096/performace-impact-of-using-setstatssamplerate-topology-stats-sample-rate
>>
>> On Wed, Jan 13, 2016 at 9:51 PM, researcher cs <prog.researcher@gmail.com
>> > wrote:
>>
>>> what is this properties used for topology.stats.sample.rate ?
>>>
>>
>>
>
Re: topology.stats.sample.rate
Posted by John Yost <ho...@gmail.com>.
Hi,
Argh, I apologize, just getting back to this. The storm default
topology.stats.sample.rate is 0.05, I set it to 1.0 for mine, and that's
when I started seeing numbers for executed/emitted/acked. I think the
sample rate needs to be higher for lower throughput installations like
mine, and maybe your's as well.
Again, sorry for the delay--hope this works for you also!
--John
On Wed, Jan 13, 2016 at 9:53 AM, Stephen Powis <sp...@salesforce.com>
wrote:
>
> http://stackoverflow.com/questions/26276096/performace-impact-of-using-setstatssamplerate-topology-stats-sample-rate
>
> On Wed, Jan 13, 2016 at 9:51 PM, researcher cs <pr...@gmail.com>
> wrote:
>
>> what is this properties used for topology.stats.sample.rate ?
>>
>
>
Re: topology.stats.sample.rate
Posted by Stephen Powis <sp...@salesforce.com>.
http://stackoverflow.com/questions/26276096/performace-impact-of-using-setstatssamplerate-topology-stats-sample-rate
On Wed, Jan 13, 2016 at 9:51 PM, researcher cs <pr...@gmail.com>
wrote:
> what is this properties used for topology.stats.sample.rate ?
>