You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by somnath <so...@gmail.com> on 2012/07/06 12:06:39 UTC
Filter and Aggregation Completion
Hi,
My requirement is
1) Read a file location and extract only those files that have "End
Date=Today's Date" as one of the headers.
2) Combine all of these files and generate a single file out of it.
When I tried with
from("file:/camel/inbox?noop=true")
.filter().method("my-filter", "readyToProcess") // Has the
logic described in requirement (1)
.aggregate(simple(ANY_LITERAL),new
MyAggregator()).completionFromBatchConsumer()
.beanRef("my-processor", "process")
.to("file:/camel/outbox?flatten=true");
The program never enters the my-processor bean class and hence the files are
not generated at all.
When removed the filter logic to the aggregate class it worked cleanly
from("file:/camel/inbox?noop=true")
.aggregate(simple(ANY_LITERAL),new
MyAggregator()).completionFromBatchConsumer()
.beanRef("my-processor", "process")
.to("file:/camel/outbox?flatten=true");
Is this an expected behaviour?
-Somnath
--
View this message in context: http://camel.465427.n5.nabble.com/Filter-and-Aggregation-Completion-tp5715604.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Filter and Aggregation Completion
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jul 6, 2012 at 1:17 PM, Pontus Ullgren <ul...@gmail.com> wrote:
> Just figured out that in newer version of Camel you set the predicate
> by calling
> from("file:/camel/inbox?noop=true")
> .aggregate().setCompletionPredicate(new Predicate() {...})
>
> Seems like the aggregator page need some TLC .
>
No, please read the documentation.
On the top of the page
http://camel.apache.org/aggregator.html
> // Pontus
>
>
> On Fri, Jul 6, 2012 at 1:04 PM, Pontus Ullgren <ul...@gmail.com> wrote:
>> Just a quick guess.
>>
>> Since you are filtering out messages from the batch the aggregator
>> will not reach a decision that the batch is completed.
>> The completeion from batch will wait for ALL message to arrive.
>> See section "Using Batch Consumer" on this page
>> http://camel.apache.org/aggregator.html
>>
>> You should probably use some other means to verify that the batch is
>> complete such as a custom PredicateAggregationCollection.
>>
>>
>> // Pontus
>>
>>
>> On Fri, Jul 6, 2012 at 12:06 PM, somnath <so...@gmail.com> wrote:
>>> Hi,
>>>
>>> My requirement is
>>> 1) Read a file location and extract only those files that have "End
>>> Date=Today's Date" as one of the headers.
>>> 2) Combine all of these files and generate a single file out of it.
>>>
>>> When I tried with
>>> from("file:/camel/inbox?noop=true")
>>> .filter().method("my-filter", "readyToProcess") // Has the
>>> logic described in requirement (1)
>>> .aggregate(simple(ANY_LITERAL),new
>>> MyAggregator()).completionFromBatchConsumer()
>>> .beanRef("my-processor", "process")
>>> .to("file:/camel/outbox?flatten=true");
>>>
>>> The program never enters the my-processor bean class and hence the files are
>>> not generated at all.
>>>
>>> When removed the filter logic to the aggregate class it worked cleanly
>>> from("file:/camel/inbox?noop=true")
>>> .aggregate(simple(ANY_LITERAL),new
>>> MyAggregator()).completionFromBatchConsumer()
>>> .beanRef("my-processor", "process")
>>> .to("file:/camel/outbox?flatten=true");
>>>
>>> Is this an expected behaviour?
>>>
>>> -Somnath
>>>
>>>
>>> --
>>> View this message in context: http://camel.465427.n5.nabble.com/Filter-and-Aggregation-Completion-tp5715604.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Re: Filter and Aggregation Completion
Posted by Pontus Ullgren <ul...@gmail.com>.
Just figured out that in newer version of Camel you set the predicate
by calling
from("file:/camel/inbox?noop=true")
.aggregate().setCompletionPredicate(new Predicate() {...})
Seems like the aggregator page need some TLC .
// Pontus
On Fri, Jul 6, 2012 at 1:04 PM, Pontus Ullgren <ul...@gmail.com> wrote:
> Just a quick guess.
>
> Since you are filtering out messages from the batch the aggregator
> will not reach a decision that the batch is completed.
> The completeion from batch will wait for ALL message to arrive.
> See section "Using Batch Consumer" on this page
> http://camel.apache.org/aggregator.html
>
> You should probably use some other means to verify that the batch is
> complete such as a custom PredicateAggregationCollection.
>
>
> // Pontus
>
>
> On Fri, Jul 6, 2012 at 12:06 PM, somnath <so...@gmail.com> wrote:
>> Hi,
>>
>> My requirement is
>> 1) Read a file location and extract only those files that have "End
>> Date=Today's Date" as one of the headers.
>> 2) Combine all of these files and generate a single file out of it.
>>
>> When I tried with
>> from("file:/camel/inbox?noop=true")
>> .filter().method("my-filter", "readyToProcess") // Has the
>> logic described in requirement (1)
>> .aggregate(simple(ANY_LITERAL),new
>> MyAggregator()).completionFromBatchConsumer()
>> .beanRef("my-processor", "process")
>> .to("file:/camel/outbox?flatten=true");
>>
>> The program never enters the my-processor bean class and hence the files are
>> not generated at all.
>>
>> When removed the filter logic to the aggregate class it worked cleanly
>> from("file:/camel/inbox?noop=true")
>> .aggregate(simple(ANY_LITERAL),new
>> MyAggregator()).completionFromBatchConsumer()
>> .beanRef("my-processor", "process")
>> .to("file:/camel/outbox?flatten=true");
>>
>> Is this an expected behaviour?
>>
>> -Somnath
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Filter-and-Aggregation-Completion-tp5715604.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Filter and Aggregation Completion
Posted by Pontus Ullgren <ul...@gmail.com>.
Just a quick guess.
Since you are filtering out messages from the batch the aggregator
will not reach a decision that the batch is completed.
The completeion from batch will wait for ALL message to arrive.
See section "Using Batch Consumer" on this page
http://camel.apache.org/aggregator.html
You should probably use some other means to verify that the batch is
complete such as a custom PredicateAggregationCollection.
// Pontus
On Fri, Jul 6, 2012 at 12:06 PM, somnath <so...@gmail.com> wrote:
> Hi,
>
> My requirement is
> 1) Read a file location and extract only those files that have "End
> Date=Today's Date" as one of the headers.
> 2) Combine all of these files and generate a single file out of it.
>
> When I tried with
> from("file:/camel/inbox?noop=true")
> .filter().method("my-filter", "readyToProcess") // Has the
> logic described in requirement (1)
> .aggregate(simple(ANY_LITERAL),new
> MyAggregator()).completionFromBatchConsumer()
> .beanRef("my-processor", "process")
> .to("file:/camel/outbox?flatten=true");
>
> The program never enters the my-processor bean class and hence the files are
> not generated at all.
>
> When removed the filter logic to the aggregate class it worked cleanly
> from("file:/camel/inbox?noop=true")
> .aggregate(simple(ANY_LITERAL),new
> MyAggregator()).completionFromBatchConsumer()
> .beanRef("my-processor", "process")
> .to("file:/camel/outbox?flatten=true");
>
> Is this an expected behaviour?
>
> -Somnath
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Filter-and-Aggregation-Completion-tp5715604.html
> Sent from the Camel - Users mailing list archive at Nabble.com.