You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@camel.apache.org by "Gordienko, Max" <Ma...@anz.com> on 2012/07/11 03:43:14 UTC

completionFromBatchConsumer() and empty messages

Hi all,

I am using a file consumer in the batch mode to consume all the files in
a directory.
I aggregate them all together using  aggregate(constant(true), new
MyAggregationStrategy()).completionFromBatchConsumer()
And it works just fine.

Now I need to be notified if there is nothing to consume (the directory
is empty). So I use sendEmptyMessageWhenIdle=true
which works just fine by itself.

But if I use both sendEmptyMessageWhenIdle and
completionFromBatchConsumer() the aggregation never completes.

Tested with camel 2.10.0
I found this completion logic in AggregateProcessor.isCompleted(String,
Exchange)

            int size = exchange.getProperty(Exchange.BATCH_SIZE, 0,
Integer.class);
            if (size > 0 && batchConsumerCounter.intValue() >= size) {
                ....
            }

In other words the aggregate will never complete if there is nothing in
the batch.

Is there an elegant way to aggregate a flow even if the batch is empty?

Thank you!
  Max

"This e-mail and any attachments to it (the "Communication") is, unless otherwise stated, confidential,  may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ National Bank Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication."

RE: completionFromBatchConsumer() and empty messages

Posted by "Gordienko, Max" <Ma...@anz.com>.

Thank you Claus,

The issue CAMEL-5437 is created.

"This e-mail and any attachments to it (the "Communication") is, unless otherwise stated, confidential,  may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ National Bank Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication."

Re: completionFromBatchConsumer() and empty messages

Posted by Claus Ibsen <cl...@gmail.com>.

Hi

No you would need to filter the message so the empty message wont get
aggregated.

But I guess since this is a new option on the batch consumers we
should take that into account in the aggregate EIP.
Do you mind log a JIRA ticket about this?


On Wed, Jul 11, 2012 at 3:43 AM, Gordienko, Max <Ma...@anz.com> wrote:
> Hi all,
>
> I am using a file consumer in the batch mode to consume all the files in
> a directory.
> I aggregate them all together using  aggregate(constant(true), new
> MyAggregationStrategy()).completionFromBatchConsumer()
> And it works just fine.
>
> Now I need to be notified if there is nothing to consume (the directory
> is empty). So I use sendEmptyMessageWhenIdle=true
> which works just fine by itself.
>
> But if I use both sendEmptyMessageWhenIdle and
> completionFromBatchConsumer() the aggregation never completes.
>
> Tested with camel 2.10.0
> I found this completion logic in AggregateProcessor.isCompleted(String,
> Exchange)
>
>             int size = exchange.getProperty(Exchange.BATCH_SIZE, 0,
> Integer.class);
>             if (size > 0 && batchConsumerCounter.intValue() >= size) {
>                 ....
>             }
>
> In other words the aggregate will never complete if there is nothing in
> the batch.
>
> Is there an elegant way to aggregate a flow even if the batch is empty?
>
> Thank you!
>   Max
>
> "This e-mail and any attachments to it (the "Communication") is, unless otherwise stated, confidential,  may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ National Bank Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication."



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

RE: completionFromBatchConsumer() and empty messages

Posted by "Gordienko, Max" <Ma...@anz.com>.

Thanks,  Pontus 

This is exactly what I am doing right now.
But it sure feels clumsy, just like using goto to break the flow...

"This e-mail and any attachments to it (the "Communication") is, unless otherwise stated, confidential,  may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ National Bank Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication."

Re: completionFromBatchConsumer() and empty messages

Posted by Pontus Ullgren <ul...@gmail.com>.

Hello,

Until this functionality is added you can use a content based router
(choice) [0] to determine if the message is a polled file or a empty
message.
If you want a notification (say by email) on a empty poll I think this
is a good way to solve it.

Something like this should work.
----------
 from("file:files/?sendEmptyMessageWhenIdle=true&noop=true")
                           .choice()
                             .when(property(Exchange.BATCH_SIZE).isNull())
                                .to("mock:notify")
                             .otherwise()
                               .to("mock:aggregate");
----------
[0] http://camel.apache.org/content-based-router.html

Best regards
Pontus Ullgren

On Wed, Jul 11, 2012 at 3:43 AM, Gordienko, Max <Ma...@anz.com> wrote:
> Hi all,
>
> I am using a file consumer in the batch mode to consume all the files in
> a directory.
> I aggregate them all together using  aggregate(constant(true), new
> MyAggregationStrategy()).completionFromBatchConsumer()
> And it works just fine.
>
> Now I need to be notified if there is nothing to consume (the directory
> is empty). So I use sendEmptyMessageWhenIdle=true
> which works just fine by itself.
>
> But if I use both sendEmptyMessageWhenIdle and
> completionFromBatchConsumer() the aggregation never completes.
>
> Tested with camel 2.10.0
> I found this completion logic in AggregateProcessor.isCompleted(String,
> Exchange)
>
>             int size = exchange.getProperty(Exchange.BATCH_SIZE, 0,
> Integer.class);
>             if (size > 0 && batchConsumerCounter.intValue() >= size) {
>                 ....
>             }
>
> In other words the aggregate will never complete if there is nothing in
> the batch.
>
> Is there an elegant way to aggregate a flow even if the batch is empty?
>
> Thank you!
>   Max
>
> "This e-mail and any attachments to it (the "Communication") is, unless otherwise stated, confidential,  may contain copyright material and is for the use only of the intended recipient. If you receive the Communication in error, please notify the sender immediately by return e-mail, delete the Communication and the return e-mail, and do not read, copy, retransmit or otherwise deal with it. Any views expressed in the Communication are those of the individual sender only, unless expressly stated to be those of Australia and New Zealand Banking Group Limited ABN 11 005 357 522, or any of its related entities including ANZ National Bank Limited (together "ANZ"). ANZ does not accept liability in connection with the integrity of or errors in the Communication, computer virus, data corruption, interference or delay arising from or in respect of the Communication."