You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Ankur Garg <an...@gmail.com> on 2015/09/23 23:12:03 UTC

Emitting Tuple to different Bolts

Hi ,

I am trying of a scenario in which I have a Spout which reads a data from a
Message Broker and emits the message as a tuple  to a Bolt for some
Processing.

Bolt post processing converts it into seperate Messages and each sub-
message has to be sent  to different Brokers which can be hosted on
different machines .

Assuming I have finite recipients (in my case there are 3 Message Brokers
for output) .

So , Bolt1 post processing can either drop the message directly to these 3
Message Brokers

Now, If I use a single Bolt here which drops the messages to these three
brokers by itself and lets say One of them fails(due to unavailability etc)
 on which I call the collector's  fail method  .

Once the fail method is called on the bolt , in my Spout fail method gets
Invoked .

Here , I believe I will have to again process the entire message again (I
have to make sure everyMessage has to be processed ) even though 2 out of 3
messages got successfully delivered .

Alternatively , even If I emit these 3 sub messages to different bolt , I
think even in that case Spout will have to process the entire message again
.

This is because I am appending a Unique Guid with the message while
emitting it first time in the spout nextTuple() method .

Is there a way to ensure that only the failed sub message should be
processed and not entire one?


Thanks
Ankur