Bindy - Avoiding bindy to discard a chunk of data when one line has incorrect format

[Aida <ai...@gmail.com>]

Hi,

I'm working with bindy to process a file. I use a splitter (something
similar to what it is said here by Claus:
http://stackoverflow.com/questions/8122748/best-strategy-for-processing-large-csv-files-in-apache-camel)
so I can send chunks of data in parallel.

The thing is that, if in a (csv) file I have one column in a line with an
incorrect format, bindy throws an Exception and the chunk is lost. I wanted
to know if it's possible to configure something so bindy discards only that
line and keeps processing the other lines that comes in the chunk (so I only
loose the line with a bad format).

Thanks in advance.


--
View this message in context: http://camel.465427.n5.nabble.com/Bindy-Avoiding-bindy-to-discard-a-chunk-of-data-when-one-line-has-incorrect-format-tp5435521p5435521.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Bindy - Avoiding bindy to discard a chunk of data when one line has incorrect format

[Aida <ai...@gmail.com>]

Hi Claus,

Thanks for your quick response. 

As you suggested we are working with chunks of size 1. The problem is that
with this approach we can´t "play" with the chunk size in order to find a
good balance between memory usage and performance (wich is very critical for
our project).


Thank you.

--
View this message in context: http://camel.465427.n5.nabble.com/Bindy-Avoiding-bindy-to-discard-a-chunk-of-data-when-one-line-has-incorrect-format-tp5435521p5440779.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Bindy - Avoiding bindy to discard a chunk of data when one line has incorrect format

[Claus Ibsen <cl...@gmail.com>]

On Fri, Jan 27, 2012 at 3:19 PM, Aida <ai...@gmail.com> wrote:
> Hi,
>
> I'm working with bindy to process a file. I use a splitter (something
> similar to what it is said here by Claus:
> http://stackoverflow.com/questions/8122748/best-strategy-for-processing-large-csv-files-in-apache-camel)
> so I can send chunks of data in parallel.
>
> The thing is that, if in a (csv) file I have one column in a line with an
> incorrect format, bindy throws an Exception and the chunk is lost. I wanted
> to know if it's possible to configure something so bindy discards only that
> line and keeps processing the other lines that comes in the chunk (so I only
> loose the line with a bad format).
>

If you split the file on a line-by-line basis (such as tokenizer with
\n), then the splitter will by default process all the lines.
And only at the end thrown an exception if one of the lines failed.

You can then configure an onException, to catch those exceptions and
just ignore by handling the exception.

Maybe something alike this which logs what line failed.

onException(Exception.class).handled(true).log("This line failed with
data ${body}");




> Thanks in advance.
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Bindy-Avoiding-bindy-to-discard-a-chunk-of-data-when-one-line-has-incorrect-format-tp5435521p5435521.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/