You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Claus Ibsen (JIRA)" <ji...@apache.org> on 2009/05/22 22:06:50 UTC
[jira] Created: (CAMEL-1640) BatchConsumer - Allow consumers to be
batch aware
BatchConsumer - Allow consumers to be batch aware
-------------------------------------------------
Key: CAMEL-1640
URL: https://issues.apache.org/activemq/browse/CAMEL-1640
Project: Apache Camel
Issue Type: New Feature
Components: camel-core
Affects Versions: 2.0-M1
Reporter: Claus Ibsen
Assignee: Claus Ibsen
Fix For: 2.0.0
Most of the consumers can handle batch but they dont decorate the Exchange with this information.
Only the file component does this.
We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
And then afterwards loop this list and emit each exchange with information about: current index out of total.
This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
In camel-core its the *file* component that should be batch aware.
Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-1640) BatchConsumer - Allow consumers to be
batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-1640.
--------------------------------
Resolution: Fixed
We got them covered now. only atom/rss is not batch consumer as they are working in a special way with how it uses it split entries to consume the next entry on the next poll. And as such its not really batch aware.
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1640) BatchConsumer - Allow consumers to
be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51869#action_51869 ]
Claus Ibsen commented on CAMEL-1640:
------------------------------------
camel-ibatis: 777941. *DONE*
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1640) BatchConsumer - Allow consumers to
be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51866#action_51866 ]
Claus Ibsen commented on CAMEL-1640:
------------------------------------
Should the batch information be set as header or properties?
{code}
// add current index and total as headers
exchange.getIn().setHeader(Exchange.BATCH_INDEX, index);
exchange.getIn().setHeader(Exchange.BATCH_SIZE, total);
exchange.getIn().setHeader(Exchange.BATCH_COMPLETE, index == total - 1);
{code}
at first I added it as properties but then I felt that as headers was where most of the others is usually set.
Any thoughts? eg headers or properties?
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1640) BatchConsumer - Allow consumers to
be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51870#action_51870 ]
Claus Ibsen commented on CAMEL-1640:
------------------------------------
camel-jpa: 777944. *DONE*
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (CAMEL-1640) BatchConsumer - Allow consumers
to be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on CAMEL-1640 started by Claus Ibsen.
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1640) BatchConsumer - Allow consumers to
be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51853#action_51853 ]
Claus Ibsen commented on CAMEL-1640:
------------------------------------
First commit in: 777808.
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1640) BatchConsumer - Allow consumers to
be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51872#action_51872 ]
Claus Ibsen commented on CAMEL-1640:
------------------------------------
Added wiki
http://camel.apache.org/batch-consumer.html
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1640) BatchConsumer - Allow consumers to
be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51867#action_51867 ]
Claus Ibsen commented on CAMEL-1640:
------------------------------------
I 2nd that. I do think they belong as properties. I changed it to headers to easier unit test camel-mail. But I will revert it to properties.
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1640) BatchConsumer - Allow consumers to
be batch aware
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51868#action_51868 ]
Claus Ibsen commented on CAMEL-1640:
------------------------------------
camel-mail: 777891. *DONE*
> BatchConsumer - Allow consumers to be batch aware
> -------------------------------------------------
>
> Key: CAMEL-1640
> URL: https://issues.apache.org/activemq/browse/CAMEL-1640
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Claus Ibsen
> Assignee: Claus Ibsen
> Fix For: 2.0.0
>
>
> Most of the consumers can handle batch but they dont decorate the Exchange with this information.
> Only the file component does this.
> We should expose a {{BatchConsumer}} interface to mark a consumer as being batch capable.
> Then this consumer should be able to round up the exchanges beforehand they are processed. This allows us to gather the total number of exchanges in the batch.
> And then afterwards loop this list and emit each exchange with information about: current index out of total.
> This also makes it much easier to support aggregator afterwards as the aggregator will be able to support BatchConsumer and then know when it has the complete batch aggregated.
> In camel-core its the *file* component that should be batch aware.
> Other components such as: mail, rss, atom, and many others can support this as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.