Can you please suggest how FailedException is working in Storm(Trident)?

[M Tarkeshwar Rao <m....@ericsson.com>]

Hi all,

Can you please suggest how FailedException is working in Storm(Trident)?
Can I send the desired Information to spout using this exception without disturbing the bolt process(Which raised the exception).

We are processing some files and those files have the data. We are using file name as tuple and using 10 files batch for processing using Trident.
We are decoding the data inside file in a bolt and getting some encode-decode errors. I want to send this error message to the spout to remove
those files from batch and replay the remaining batch.

Another strategy we planned to store the failed information as log on zookeeper and get the log from there . Is this strategy is better or any risk in it?

Please suggest if you can any other way to do it.

Regards
Tarkeshwar

Re: Can you please suggest how FailedException is working in Storm(Trident)?

[Abhishek Bhattacharjee <ab...@gmail.com>]

It is not a good idea to store anything on zookeeper other than the
metadata. You can use a centralized data-store for storing your logs.

Now for the spout thing , why do you want to remove the files using a spout
you can do that using another bolt say "removebolt" which is meant for
removing the files from the batch. You can catch the encode-decode
exception and do the removal in the catch block. No need to traceback to
your spout.
Spout is only meant for fetching and supplying data.
On Mar 24, 2014 2:35 PM, "M Tarkeshwar Rao" <m....@ericsson.com>
wrote:

>  Hi all,
>
>
>
> Can you please suggest how FailedException is working in Storm(Trident)?
>
> Can I send the desired Information to spout using this exception without
> disturbing the bolt process(Which raised the exception).
>
>
>
> We are processing some files and those files have the data. We are using
> file name as tuple and using 10 files batch for processing using Trident.
>
> We are decoding the data inside file in a bolt and getting some
> encode-decode errors. I want to send this error message to the spout to
> remove
>
> those files from batch and replay the remaining batch.
>
>
>
> Another strategy we planned to store the failed information as log on
> zookeeper and get the log from there . Is this strategy is better or any
> risk in it?
>
>
>
> Please suggest if you can any other way to do it.
>
>
>
> Regards
>
> Tarkeshwar
>