You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Adrien Carreira <ac...@reportlinker.com> on 2016/05/02 15:08:02 UTC
Spout Questions
Hi there,
Dont't know if I'm on the right place.. But let's try.
I'm build a Topology, And I've a spout plugged on Redis.
My question is, when the topology is active, Why the nextTuple() method
isn't call when ack() method is called.
Meaning, I've about 10k acking message without a nextTuple() called...
So what going is : nextTuple is called to emit 30000k message, stops,
acking is called to ack all message without calling nextTuple to refeed the
topoogy....
What can be the problem ?
Thanks for your feedbacks and sorry for my bad english.
Re: Spout Questions
Posted by Adrien Carreira <ac...@reportlinker.com>.
Thank you for the feedback.
I've juste switched back to 0.10 and I don't have the issue.
nextTuple is quite fast, I'm handleling a buffer of Tuple, when the buffer
is empty, I'm loading data from redis.
Ack method is deleting data on redis using pipeline to be fast.
It's strange that in the 0.10 I reach the buffer size and note in the 1.0
I developping a web crawler, and to be fast I need to emit a constant
quantity of Tuple.
Adrien
Le lundi 2 mai 2016, P. Taylor Goetz <pt...@gmail.com> a écrit :
> nextTuple(), ack(), and fail() are all called by the same thread.
> nextTuple() should be fast, so you probably only want to emit one or a
> handful of tuples. Emitting a huge number of tuples in the nextTuple()
> method is what’s causing your problem.
>
> -Taylor
>
> > On May 2, 2016, at 9:08 AM, Adrien Carreira <aca@reportlinker.com
> <javascript:;>> wrote:
> >
> > Hi there,
> >
> > Dont't know if I'm on the right place.. But let's try.
> >
> > I'm build a Topology, And I've a spout plugged on Redis.
> >
> > My question is, when the topology is active, Why the nextTuple() method
> isn't call when ack() method is called.
> >
> > Meaning, I've about 10k acking message without a nextTuple() called...
> >
> > So what going is : nextTuple is called to emit 30000k message, stops,
> acking is called to ack all message without calling nextTuple to refeed the
> topoogy....
> >
> > What can be the problem ?
> >
> >
> > Thanks for your feedbacks and sorry for my bad english.
> >
>
>
Re: Spout Questions
Posted by "P. Taylor Goetz" <pt...@gmail.com>.
nextTuple(), ack(), and fail() are all called by the same thread. nextTuple() should be fast, so you probably only want to emit one or a handful of tuples. Emitting a huge number of tuples in the nextTuple() method is what’s causing your problem.
-Taylor
> On May 2, 2016, at 9:08 AM, Adrien Carreira <ac...@reportlinker.com> wrote:
>
> Hi there,
>
> Dont't know if I'm on the right place.. But let's try.
>
> I'm build a Topology, And I've a spout plugged on Redis.
>
> My question is, when the topology is active, Why the nextTuple() method isn't call when ack() method is called.
>
> Meaning, I've about 10k acking message without a nextTuple() called...
>
> So what going is : nextTuple is called to emit 30000k message, stops, acking is called to ack all message without calling nextTuple to refeed the topoogy....
>
> What can be the problem ?
>
>
> Thanks for your feedbacks and sorry for my bad english.
>