Aggregator clarification

[mikaelfj <mi...@gmail.com>]

Hi,

We are in the process of using the Aggregator component in our integration,
and I need some clarification of whether my understanding of its persistence
is correct.

We are reading files from a folder structure like this:

user1/inv1/..
user1/inv2/..

where all files found in e.g. user1/inv1 has to be grouped and handled
together - which is why we use the aggregator to perform the grouping of the
files.

We are reading the folder structure from a ftp-server, and originally I had
expected, that if something went wrong in the aggregator or later in the
route, the files would still present in the folder structure (like it would
in a non-aggregator route).

But I can see that this is not the case, and after reading chapter 8 of
Camel in Action, I assume that what I need to do is to use the HawtDb as a
recoverable aggregation repository and this way add persistence/recovery to
the route.

Is this the correct approach or can the route, by other means, be made to
work so it leaves the files in the folder structure in case it is unable to
complete the route?

/Mikael 

--
View this message in context: http://camel.465427.n5.nabble.com/Aggregator-clarification-tp5119201p5119201.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Aggregator clarification

[Claus Ibsen <cl...@gmail.com>]

Hi

Yes some sort of persistence is needed as the aggregator EIP is stateful.

However since the source of the messages is already persistence (eg
files), you may aggregate them for example by moving them to a special
"aggregate directory", or leave them as is. And only store meta-data
about the aggregated files in the aggregated Exchange. For example the
names of the files.

Then the outgoing Exchange from the aggregator just contains a list of
file names, and then you can process the Exchange by accessing the
list of file names, and load the file content as needed. And when you
are done, you can manually delete or move the files to a backup
directory or whatever.







On Wed, Jan 4, 2012 at 10:53 AM, mikaelfj <mi...@gmail.com> wrote:
> Hi,
>
> We are in the process of using the Aggregator component in our integration,
> and I need some clarification of whether my understanding of its persistence
> is correct.
>
> We are reading files from a folder structure like this:
>
> user1/inv1/..
> user1/inv2/..
>
> where all files found in e.g. user1/inv1 has to be grouped and handled
> together - which is why we use the aggregator to perform the grouping of the
> files.
>
> We are reading the folder structure from a ftp-server, and originally I had
> expected, that if something went wrong in the aggregator or later in the
> route, the files would still present in the folder structure (like it would
> in a non-aggregator route).
>
> But I can see that this is not the case, and after reading chapter 8 of
> Camel in Action, I assume that what I need to do is to use the HawtDb as a
> recoverable aggregation repository and this way add persistence/recovery to
> the route.
>
> Is this the correct approach or can the route, by other means, be made to
> work so it leaves the files in the folder structure in case it is unable to
> complete the route?
>
> /Mikael
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Aggregator-clarification-tp5119201p5119201.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/