Re: How Storm Process tuples atleast once?

[Javier Gonzalez <ja...@gmail.com>]

Hi,

If it's a custom Spout (i.e. you wrote it) it's completely up to you to
re-emit in the event of a failed tuple. You get whatever ID you sent down
the topology when you emitted from the spout, so make sure you are able to
somehow get the message back from that ID to re-emit.

Regards,
JG

On Mon, May 18, 2015 at 7:42 AM, prasad ch <ch...@outlook.com> wrote:

>
>
> Thanks for your *valuable* responses !
>
>
> Here when ever tuple is failed ,fail()  method of spout will be called
> with message id.
> Here when ever any tuple enters into fail method , who will replay that
> tuple ?it is automatically done by storm or
> developer need to emit from fail() method ! please help me !
>
>
>
> Thanks!
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> > Date: Mon, 18 May 2015 10:15:57 +0200
> > From: mjsax@informatik.hu-berlin.de
> > To: user@storm.apache.org
> > Subject: Re: How Storm Process tuples atleast once?
>
> >
> > I just want to add, that duplicates in the result are easily possible.
> >
> > Let's say, the last bolt (ie, a sink), writes some tuples to a file
> > before acking it. If the write is successful and the bolt fails for some
> > reason before the ack goes through, the tuple will be replayed and
> > written to the file a second time.
> >
> >
> > -Matthias
> >
> >
> > On 05/18/2015 07:44 AM, Jake K. Dodd wrote:
> > > 'At least once' partially (in some cases, entirely) depends on your
> > > spout implementation.
> > >
> > > With the following topology:
> > >
> > > Spout --> Bolt A --> Bolt B --> Bolt C
> > >
> > > assume that Spout is Storm's officially-maintained Kafka spout. If
> Spout
> > > emits a tuple, and subsequently that tuple times out during the
> > > execution of Bolt C, Spout will replay the tuple--from the beginning of
> > > the topology. Bolts A & B will then process the tuple a second time.
> > > Thus, 'at least once.' Equivalently, if Bolt B explicitly fails the
> > > tuple, then Spout will replay the tuple and Bolt A will process the
> > > tuple a second time.
> > >
> > > If the operations in Bolt A,B are not idempotent, you might be in
> > > trouble if Bolt C fails the tuple.
> > >
> > > Alternatively, you might have a custom Spout that silently ignores a
> > > failed tuple, in which case Storm will not provide you an 'exactly
> once'
> > > guarantee; it's up to the Spout to replay a tuple (or not) when it's
> > > notified of a failed tuple tree.
> > >
> > > I don't know of any scenarios wherein a tuple will successfully
> traverse
> > > the entire tuple tree and then be replayed by the spout. If there is
> > > such a case, I'd like to hear about it.
> > >
> > > Bottom line is that if you're unable to make your bolts idempotent, and
> > > this concerns you, look at Trident to get exactly-once guarantees.
> There
> > > are all sorts of situations whereby you will encounter failed tuples,
> > > some of which are entirely out of your control (e.g. network failures).
> > >
> > > Cheers
> > >
> > > Jake K Dodd
> > >
> > > On May 17, 2015, at 22:15, prasad ch <ch.kmvprasad@outlook.com
> > > <ma...@outlook.com>> wrote:
> > >
> > >> Hi,
> > >>
> > >> In storm document they mentioned ,storm will process tuples at least
> > >> once ,means it may be chance of processing same tuple more than once!
> > >>
> > >>
> > >> *Can you explain what are the possible scenarios storm will Process
> > >> more than Once?*
> > >>
> > >>
> > >> am successfully setup storm cluster, here am using centOs my linux
> > >> machines contains python 2.6 and daemontools inbuilt.
> > >>
> > >>
> > >>
> > >> Here am failed to start storm cluster setup under *daemon tools*
> > >> ( http://cr.yp.to/daemontools.html )
> > >> ,*please help me* how to write script and hoe to start under it.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> Reagrd's
> > >>
> > >> Prasad.ch <http://Prasad.ch>
> >
>



-- 
Javier González Nicolini