Camel aggregation logic fails in case of two camel nodes

[Gnanaguru S <gn...@wipro.com>]

Hi,

I have a use case where I have used splitter and aggregation patterns.  

The diagram below shows the message flow. I have a message with ten trades:
it reaches the splitter -> gets split into ten messages -> each message gets
enriched -> after enrichment, the messages are aggregated into single
message and sent to the output queue. 


<http://camel.465427.n5.nabble.com/file/n5730994/camel-aggregation.png> 
Figure 1: Use case description

I am using the bread-crumb id as the correlation id for the messages. It
works perfectly fine for a single camel instance. When I test the same
scenario with two or more camel instances simultaneously at a time, this
aggregation logic doesn't work because the messages are consumed by many
camel node. Logically I am not able to aggregate. Hope its understandable.

What is the advice for me ? If I want to do a aggregation in a distributed
or clustered scenario ? 

Is there any pattern which can solve this problem ?

Please help.

Regards
Guru
@gnanagurus





--
View this message in context: http://camel.465427.n5.nabble.com/Camel-aggregation-logic-fails-in-case-of-two-camel-nodes-tp5730994.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Camel aggregation logic fails in case of two camel nodes

[Claus Ibsen <cl...@gmail.com>]

Hi

You need to use a persistent aggregation repository implementation.

There is a little detail here
http://camel.apache.org/aggregator2

Notice that hawtdb is being @deprecated and consider using
camel-leveldb instead. Or any other custom implementation.
There is a SQL based implementation as well out of the box.

And we should have others coming in the future based on other
technologies. I think someone mentioned voldermort, and you can
possible also use hazelcast (distributed memory) or other techs.

As usual we love contributions, so if people build custom agg repos
for other techs, then we love having them out of the box at ASF Camel.


On Wed, Apr 17, 2013 at 8:07 AM, Gnanaguru S
<gn...@wipro.com> wrote:
> Hi,
>
> I have a use case where I have used splitter and aggregation patterns.
>
> The diagram below shows the message flow. I have a message with ten trades:
> it reaches the splitter -> gets split into ten messages -> each message gets
> enriched -> after enrichment, the messages are aggregated into single
> message and sent to the output queue.
>
>
> <http://camel.465427.n5.nabble.com/file/n5730994/camel-aggregation.png>
> Figure 1: Use case description
>
> I am using the bread-crumb id as the correlation id for the messages. It
> works perfectly fine for a single camel instance. When I test the same
> scenario with two or more camel instances simultaneously at a time, this
> aggregation logic doesn't work because the messages are consumed by many
> camel node. Logically I am not able to aggregate. Hope its understandable.
>
> What is the advice for me ? If I want to do a aggregation in a distributed
> or clustered scenario ?
>
> Is there any pattern which can solve this problem ?
>
> Please help.
>
> Regards
> Guru
> @gnanagurus
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Camel-aggregation-logic-fails-in-case-of-two-camel-nodes-tp5730994.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen