You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by kodcanavari <ko...@gmail.com> on 2009/10/15 15:01:03 UTC
How to aggregate all messages into a single message?
Hi,
I want to use an aggregator in order to aggregate all messages comming.
On my route i'm making a split with a splitter , then i process every
messages in parallel
and then i want to aggregate them all into a single file.
Here is my snippet :
this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
LdapAggregationStrategy()).to(TO_FOLDER);
and inside LdapModifyProcessor() i do:
exchange.getOut().setHeader("type", "LDIFRecord");
but it does not work.
can anyone tellme how can i let every messages go thru aggregator?
regards.
--
View this message in context: http://www.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p25908234.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to aggregate all messages into a single message?
Posted by Claus Ibsen <cl...@gmail.com>.
Ah it should haven been without
together *with out* the aggregator.
On Thu, Oct 15, 2009 at 3:05 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> The splitter has build in join so you can do that all together with
> the aggregator.
>
> See the "Split aggregate request/reply sample" in
> http://camel.apache.org/splitter.html
>
>
> On Thu, Oct 15, 2009 at 3:01 PM, kodcanavari <ko...@gmail.com> wrote:
>>
>> Hi,
>>
>> I want to use an aggregator in order to aggregate all messages comming.
>> On my route i'm making a split with a splitter , then i process every
>> messages in parallel
>> and then i want to aggregate them all into a single file.
>>
>> Here is my snippet :
>>
>>
>> this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
>> LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
>> LdapAggregationStrategy()).to(TO_FOLDER);
>>
>>
>> and inside LdapModifyProcessor() i do:
>>
>> exchange.getOut().setHeader("type", "LDIFRecord");
>>
>> but it does not work.
>>
>> can anyone tellme how can i let every messages go thru aggregator?
>>
>> regards.
>>
>>
>>
>> --
>> View this message in context: http://www.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p25908234.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How to aggregate all messages into a single message?
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
The splitter has build in join so you can do that all together with
the aggregator.
See the "Split aggregate request/reply sample" in
http://camel.apache.org/splitter.html
On Thu, Oct 15, 2009 at 3:01 PM, kodcanavari <ko...@gmail.com> wrote:
>
> Hi,
>
> I want to use an aggregator in order to aggregate all messages comming.
> On my route i'm making a split with a splitter , then i process every
> messages in parallel
> and then i want to aggregate them all into a single file.
>
> Here is my snippet :
>
>
> this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
> LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
> LdapAggregationStrategy()).to(TO_FOLDER);
>
>
> and inside LdapModifyProcessor() i do:
>
> exchange.getOut().setHeader("type", "LDIFRecord");
>
> but it does not work.
>
> can anyone tellme how can i let every messages go thru aggregator?
>
> regards.
>
>
>
> --
> View this message in context: http://www.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p25908234.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How to aggregate all messages into a single message?
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Thanks for sharing your solution.
Looks nice with separate routes for the aggregation. Makes it easier
to test as well :)
On Mon, Nov 2, 2009 at 9:18 PM, kodcanavari <ko...@gmail.com> wrote:
>
> I resolved my problem by this way:
>
> this.from(m_from_folder_url)
> .split(new SplitExpression())
> .process(new Processor())
> .choice()
> .when(header("type").isEqualTo("add"))
> .to("direct:addEntry")
> .when(header("type").isEqualTo("modify"))
> .to("direct:modifyEntry");
>
> from("direct:addEntry").aggregate(header("add"), new
> AggregationStrategy()).to(m_to_add_folder_url);
> from("direct:modifyEntry").aggregate(header("modify"), new
> AggregationStrategy()).to(m_to_modify_folder_url);
>
> so this is a route which:
> - reads a file
> - splits it's entities
> - processes each entry separately
> - according to their nature aggregating processed entries to the
> appropriate file.
>
>
>
> kodcanavari wrote:
>>
>> Hi,
>>
>> I want to use an aggregator in order to aggregate all messages comming.
>> On my route i'm making a split with a splitter , then i process every
>> messages in parallel
>> and then i want to aggregate them all into a single file.
>>
>> Here is my snippet :
>>
>>
>> this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
>> LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
>> LdapAggregationStrategy()).to(TO_FOLDER);
>>
>>
>> and inside LdapModifyProcessor() i do:
>>
>> exchange.getOut().setHeader("type", "LDIFRecord");
>>
>> but it does not work.
>>
>> can anyone tellme how can i let every messages go thru aggregator?
>>
>> regards.
>>
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p26157808.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: How to aggregate all messages into a single message?
Posted by kodcanavari <ko...@gmail.com>.
I resolved my problem by this way:
this.from(m_from_folder_url)
.split(new SplitExpression())
.process(new Processor())
.choice()
.when(header("type").isEqualTo("add"))
.to("direct:addEntry")
.when(header("type").isEqualTo("modify"))
.to("direct:modifyEntry");
from("direct:addEntry").aggregate(header("add"), new
AggregationStrategy()).to(m_to_add_folder_url);
from("direct:modifyEntry").aggregate(header("modify"), new
AggregationStrategy()).to(m_to_modify_folder_url);
so this is a route which:
- reads a file
- splits it's entities
- processes each entry separately
- according to their nature aggregating processed entries to the
appropriate file.
kodcanavari wrote:
>
> Hi,
>
> I want to use an aggregator in order to aggregate all messages comming.
> On my route i'm making a split with a splitter , then i process every
> messages in parallel
> and then i want to aggregate them all into a single file.
>
> Here is my snippet :
>
>
> this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
> LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
> LdapAggregationStrategy()).to(TO_FOLDER);
>
>
> and inside LdapModifyProcessor() i do:
>
> exchange.getOut().setHeader("type", "LDIFRecord");
>
> but it does not work.
>
> can anyone tellme how can i let every messages go thru aggregator?
>
> regards.
>
>
>
>
--
View this message in context: http://old.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p26157808.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to aggregate all messages into a single message?
Posted by kodcanavari <ko...@gmail.com>.
Thanks a lot dulanov, it worked :)
but can you tell me what exactly end() do and why i cant use the aggregator
class ?
dulanov wrote:
>
> Try to use this.from(FROM_FOLDER).split(new LdapEntitySplitExpression(),
> new
> LdapAggregationStrategy()).process(new
> LdapModifyProcessor()).end().to(TO_FOLDER);
>
> On Thu, Oct 15, 2009 at 5:01 PM, kodcanavari <ko...@gmail.com>
> wrote:
>
>>
>> Hi,
>>
>> I want to use an aggregator in order to aggregate all messages comming.
>> On my route i'm making a split with a splitter , then i process every
>> messages in parallel
>> and then i want to aggregate them all into a single file.
>>
>> Here is my snippet :
>>
>>
>> this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
>> LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
>> LdapAggregationStrategy()).to(TO_FOLDER);
>>
>>
>> and inside LdapModifyProcessor() i do:
>>
>> exchange.getOut().setHeader("type", "LDIFRecord");
>>
>> but it does not work.
>>
>> can anyone tellme how can i let every messages go thru aggregator?
>>
>> regards.
>>
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p25908234.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context: http://www.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p25909223.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to aggregate all messages into a single message?
Posted by kodcanavari <ko...@gmail.com>.
this.from(FROM_FOLDER)
.split(new LdapEntitySplitExpression(), new LdapAggregationStrategy())
.process(new LdapModifyProcessor())
.end()
.to(TO_FOLDER);
it works and aggregates processed messages into the same file but i want to
complicate a little bit.
What i want to do is to split a file from a file endpoint, process each
entry separately and then according to the entry content, i want to
aggregate them into different files. So i wrote the code below but in this
case the aggregation doesn't work, the outputs are in separate files.
this.from(m_from_folder_url)
.split(new LdapEntitySplitExpression(),new
LdapAggregationStrategy())
.process(new LdapModifyProcessor(hMap))
.choice()
.when(header("type").isEqualTo("add"))
.to(m_to_folder_url)
.when(header("type").isEqualTo("modify"))
.to(m_to_folder_url)
.end();
So does anyone have an idea to solve it?
Thanks.
dulanov wrote:
>
> Try to use this.from(FROM_FOLDER).split(new LdapEntitySplitExpression(),
> new
> LdapAggregationStrategy()).process(new
> LdapModifyProcessor()).end().to(TO_FOLDER);
>
> On Thu, Oct 15, 2009 at 5:01 PM, kodcanavari <ko...@gmail.com>
> wrote:
>
>>
>> Hi,
>>
>> I want to use an aggregator in order to aggregate all messages comming.
>> On my route i'm making a split with a splitter , then i process every
>> messages in parallel
>> and then i want to aggregate them all into a single file.
>>
>> Here is my snippet :
>>
>>
>> this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
>> LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
>> LdapAggregationStrategy()).to(TO_FOLDER);
>>
>>
>> and inside LdapModifyProcessor() i do:
>>
>> exchange.getOut().setHeader("type", "LDIFRecord");
>>
>> but it does not work.
>>
>> can anyone tellme how can i let every messages go thru aggregator?
>>
>> regards.
>>
>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p25908234.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context: http://old.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p26155433.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to aggregate all messages into a single message?
Posted by "kraythe ." <kr...@gmail.com>.
Its easy to use a custom aggregator than try to do with DSL.
See http://camel.apache.org/aggregator2
As an example:
public class ListAggregationStrategy<T> implements AggregationStrategy {
/** The comparator used to keep the list sorted if any. */
final Comparator<T> comparator;
/**
* @param comparator The comparator used to keep the list sorted if any;
if this is null then the list will not be kept sorted.
*/
public ListAggregationStrategy(final Comparator<T> comparator) {
this.comparator = comparator;
}
@Override
public Exchange aggregate(final Exchange aggregatedExchange, final
Exchange newExchange) {
final T newBody = (T) newExchange.getIn().getBody();
List<T> exchangeBodies = null;
if (aggregatedExchange == null) {
exchangeBodies = new LinkedList<>();
exchangeBodies.add(newBody);
newExchange.getIn().setBody(exchangeBodies);
return newExchange;
}
exchangeBodies = aggregatedExchange.getIn().getBody(List.class);
exchangeBodies.add(newBody);
if (null != this.comparator) Collections.sort(exchangeBodies,
this.comparator);
return aggregatedExchange;
}
}
*Robert Simmons Jr. MSc. - Lead Java Architect @ EA*
*Author of: Hardcore Java (2003) and Maintainable Java (2012)*
*LinkedIn: **http://www.linkedin.com/pub/robert-simmons/40/852/a39
<http://www.linkedin.com/pub/robert-simmons/40/852/a39>*
On Tue, Feb 25, 2014 at 4:24 AM, Jens Breitenstein <ma...@j-b-s.de>wrote:
> Hi
>
> we aggragate all messages having the same key like this:
>
> .aggregate(new GroovyExpression("exchange.in.body.InstrumentID"),
> new QuoteAggregationStrategy())
>
> and get a List as result ("isStoreAsBodyOnCompletion() returns true").
>
> The QuoteAggregationStrategy is required to implement the abstract method
> <T> getValue(final Exchange exchange) to convert the message to a specific
> type.
> In case you do not have a aggregateble key you can have a look at
> AbstractListAggregationStrategy and all derived classes, maybe one fits
> because you need some state to keep track about the number of exchanges
> combined and the desired chunk size. Maybe you create your own aggregator
> derived from AbstractListAggregationStrategy and override this method:
>
> public Exchange aggregate(Exchange oldExchange, Exchange newExchange)
>
> ?
>
> Jens
>
>
>
> Am 25.02.14 10:11, schrieb tharangage:
>
> I am having another issue based on same solution given by "dulanov".
>> Is it possible to group N lines together?
>>
>> I come up with solution as follow without aggregation, but it is better
>> if i
>> can use aggregation as describe in this post. Can anyone please help on
>> this?
>>
>> from("file:/temp/test").split().tokenize("\n", 100).process(new
>> MyProcessor()).to("file:/home/test/dest?fileExist=Append");
>>
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.
>> com/How-to-aggregate-all-messages-into-a-single-
>> message-tp477603p5747917.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
>
Re: How to aggregate all messages into a single message?
Posted by Jens Breitenstein <ma...@j-b-s.de>.
Hi
we aggragate all messages having the same key like this:
.aggregate(new
GroovyExpression("exchange.in.body.InstrumentID"), new
QuoteAggregationStrategy())
and get a List as result ("isStoreAsBodyOnCompletion() returns true").
The QuoteAggregationStrategy is required to implement the abstract
method <T> getValue(final Exchange exchange) to convert the message to a
specific type.
In case you do not have a aggregateble key you can have a look at
AbstractListAggregationStrategy and all derived classes, maybe one fits
because you need some state to keep track about the number of exchanges
combined and the desired chunk size. Maybe you create your own
aggregator derived from AbstractListAggregationStrategy and override
this method:
public Exchange aggregate(Exchange oldExchange, Exchange newExchange)
?
Jens
Am 25.02.14 10:11, schrieb tharangage:
> I am having another issue based on same solution given by "dulanov".
> Is it possible to group N lines together?
>
> I come up with solution as follow without aggregation, but it is better if i
> can use aggregation as describe in this post. Can anyone please help on
> this?
>
> from("file:/temp/test").split().tokenize("\n", 100).process(new
> MyProcessor()).to("file:/home/test/dest?fileExist=Append");
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/How-to-aggregate-all-messages-into-a-single-message-tp477603p5747917.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to aggregate all messages into a single message?
Posted by tharangage <th...@gmail.com>.
I am having another issue based on same solution given by "dulanov".
Is it possible to group N lines together?
I come up with solution as follow without aggregation, but it is better if i
can use aggregation as describe in this post. Can anyone please help on
this?
from("file:/temp/test").split().tokenize("\n", 100).process(new
MyProcessor()).to("file:/home/test/dest?fileExist=Append");
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-aggregate-all-messages-into-a-single-message-tp477603p5747917.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to aggregate all messages into a single message?
Posted by Dmitry Ulanov <du...@gmail.com>.
Try to use this.from(FROM_FOLDER).split(new LdapEntitySplitExpression(), new
LdapAggregationStrategy()).process(new
LdapModifyProcessor()).end().to(TO_FOLDER);
On Thu, Oct 15, 2009 at 5:01 PM, kodcanavari <ko...@gmail.com> wrote:
>
> Hi,
>
> I want to use an aggregator in order to aggregate all messages comming.
> On my route i'm making a split with a splitter , then i process every
> messages in parallel
> and then i want to aggregate them all into a single file.
>
> Here is my snippet :
>
>
> this.from(FROM_FOLDER).split(new LdapEntitySplitExpression()).process(new
> LdapModifyProcessor()).aggregate(header("LDIFRecord"),new
> LdapAggregationStrategy()).to(TO_FOLDER);
>
>
> and inside LdapModifyProcessor() i do:
>
> exchange.getOut().setHeader("type", "LDIFRecord");
>
> but it does not work.
>
> can anyone tellme how can i let every messages go thru aggregator?
>
> regards.
>
>
>
> --
> View this message in context:
> http://www.nabble.com/How-to-aggregate-all-messages-into-a-single-message--tp25908234p25908234.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>