You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Walid Aljoby <wa...@yahoo.com> on 2016/07/16 10:37:48 UTC
Processing guarantee in Apache Storm
Hi Everyone,
I have fundamental questions about processing guarantee in Apache Storm. So Anyone please can help for the following:
1- Animation or clear tutorial with example show the anchoring mechanism..2- Any topology code already implemented the guarantee processing function.
Many thanks,
--Regards
WA
Re: Processing guarantee in Apache Storm
Posted by Walid Aljoby <wa...@yahoo.com>.
Many thanks Andrew for details reply. Its very helpful tutorials.
---RegardsWalid
From: Andrew Montalenti <an...@parsely.com>
To: user@storm.apache.org; Walid Aljoby <wa...@yahoo.com>
Sent: Sunday, July 24, 2016 10:50 AM
Subject: Re: Processing guarantee in Apache Storm
Hey Walid,
I have an explanation of tuple trees (and anchoring) in my PyCon 2015 presentation on streamparse + Storm, starting at minute 11. You can find that here:
https://youtu.be/ja4Qj9-l6WQ?t=11m
As for your second request, any topology that does the following will implement "guaranteed processing":
1. assigns tuple IDs to tuples in the spout2. "anchors" tuples in the bolts at every point where a tuple is emitted, simply by passing it to the emit method3. ack's and/or fail's tuples at every bolt stage of processing
This is explained in (good) detail in this blog post on fault tolerant processing with Storm.
Most topologies that you see in the wild will do this, since it's such a big part of Storm's benefit.
Hope that helps,
--
Andrew Montalenti | CTO
On Sat, Jul 16, 2016 at 6:37 AM, Walid Aljoby <wa...@yahoo.com> wrote:
Hi Everyone,
I have fundamental questions about processing guarantee in Apache Storm. So Anyone please can help for the following:
1- Animation or clear tutorial with example show the anchoring mechanism..2- Any topology code already implemented the guarantee processing function.
Many thanks,
--Regards
WA
Re: Processing guarantee in Apache Storm
Posted by Andrew Montalenti <an...@parsely.com>.
Hey Walid,
I have an explanation of tuple trees (and anchoring) in my PyCon 2015
presentation on streamparse + Storm, starting at minute 11. You can find
that here:
https://youtu.be/ja4Qj9-l6WQ?t=11m
As for your second request, any topology that does the following will
implement "guaranteed processing":
1. assigns tuple IDs to tuples in the spout
2. "anchors" tuples in the bolts at every point where a tuple is emitted,
simply by passing it to the emit method
3. ack's and/or fail's tuples at every bolt stage of processing
This is explained in (good) detail in this blog post on fault tolerant
processing with Storm
<https://bryantsai.com/fault-tolerant-message-processing-in-storm/>.
Most topologies that you see in the wild will do this, since it's such a
big part of Storm's benefit.
Hope that helps,
--
Andrew Montalenti | CTO
<http://www.parsely.com/?utm_source=Signature&utm_medium=Andrew&utm_campaign=Signature>
On Sat, Jul 16, 2016 at 6:37 AM, Walid Aljoby <wa...@yahoo.com>
wrote:
> Hi Everyone,
>
> I have fundamental questions about processing guarantee in Apache Storm.
> So Anyone please can help for the following:
>
> 1- Animation or clear tutorial with example show the anchoring mechanism..
> 2- Any topology code already implemented the guarantee processing function.
>
>
> Many thanks,
>
> --
> Regards
>
> WA
>