You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Ankur Garg <an...@gmail.com> on 2015/09/23 23:12:03 UTC
Emitting Tuple to different Bolts
Hi ,
I am trying of a scenario in which I have a Spout which reads a data from a
Message Broker and emits the message as a tuple to a Bolt for some
Processing.
Bolt post processing converts it into seperate Messages and each sub-
message has to be sent to different Brokers which can be hosted on
different machines .
Assuming I have finite recipients (in my case there are 3 Message Brokers
for output) .
So , Bolt1 post processing can either drop the message directly to these 3
Message Brokers
Now, If I use a single Bolt here which drops the messages to these three
brokers by itself and lets say One of them fails(due to unavailability etc)
on which I call the collector's fail method .
Once the fail method is called on the bolt , in my Spout fail method gets
Invoked .
Here , I believe I will have to again process the entire message again (I
have to make sure everyMessage has to be processed ) even though 2 out of 3
messages got successfully delivered .
Alternatively , even If I emit these 3 sub messages to different bolt , I
think even in that case Spout will have to process the entire message again
.
This is because I am appending a Unique Guid with the message while
emitting it first time in the spout nextTuple() method .
Is there a way to ensure that only the failed sub message should be
processed and not entire one?
Thanks
Ankur