You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Ron Gavlin (JIRA)" <ji...@apache.org> on 2009/09/30 07:41:00 UTC
[jira] Created: (CAMEL-2047) camel-jms async consumer needs dynamic
mechanism to throttle message consumption
camel-jms async consumer needs dynamic mechanism to throttle message consumption
--------------------------------------------------------------------------------
Key: CAMEL-2047
URL: https://issues.apache.org/activemq/browse/CAMEL-2047
Project: Apache Camel
Issue Type: Improvement
Components: camel-jms
Affects Versions: 2.0.0
Reporter: Ron Gavlin
camel-jms should provide an async consumer that can throttle consumption of incoming messages. This is necessary in scenarios where a large backlog of messages exists on the input queue. In this case, it is likely that an async consumer will overload camel with message exchanges resulting in problems. A dynamic throttling mechanism on the async camel-jms consumer is required to avoid this problem.
One possible implementation might be as follows:
First, add a property "maxPendingAsyncExchanges" on the async camel-jms consumer endpoint. Then add support in the endpoint to track the total number of pending/in-progress exchanges, i.e., the number of exchanges for which no SUCCESS/ERROR acknowledgement has been returned. When this value reaches "maxPendingAsyncExchanges", then the endpoint would stop its Spring JMS DefaultMessageListenerContainer. Once the number of "pending" exchanges drops below a percentage of the "maxPendingAsyncExchanges" threshold, say "reconnectPercentOfMax", the Spring DMLC would be restarted.
The CXF implementation of this capability is described in CXF-2002, CXF-2342, and CXF-2391. CXF has properties maxSuspendedContinuations and reconnectPercentOfMax to control the throttling behavior.
This is the type of dynamic throttling I envisioned in Camel. Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-2047) camel-jms async consumer needs
dynamic mechanism to throttle message consumption
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-2047.
--------------------------------
Resolution: Fixed
Fix Version/s: 2.1.0
Assignee: Claus Ibsen
> camel-jms async consumer needs dynamic mechanism to throttle message consumption
> --------------------------------------------------------------------------------
>
> Key: CAMEL-2047
> URL: https://issues.apache.org/activemq/browse/CAMEL-2047
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-jms
> Affects Versions: 2.0.0
> Reporter: Ron Gavlin
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
>
> camel-jms should provide an async consumer that can throttle consumption of incoming messages. This is necessary in scenarios where a large backlog of messages exists on the input queue. In this case, it is likely that an async consumer will overload camel with message exchanges resulting in problems. A dynamic throttling mechanism on the async camel-jms consumer is required to avoid this problem.
> One possible implementation might be as follows:
> First, add a property "maxPendingAsyncExchanges" on the async camel-jms consumer endpoint. Then add support in the endpoint to track the total number of pending/in-progress exchanges, i.e., the number of exchanges for which no SUCCESS/ERROR acknowledgement has been returned. When this value reaches "maxPendingAsyncExchanges", then the endpoint would stop its Spring JMS DefaultMessageListenerContainer. Once the number of "pending" exchanges drops below a percentage of the "maxPendingAsyncExchanges" threshold, say "reconnectPercentOfMax", the Spring DMLC would be restarted.
> The CXF implementation of this capability is described in CXF-2002, CXF-2342, and CXF-2391. CXF has properties maxSuspendedContinuations and reconnectPercentOfMax to control the throttling behavior.
> This is the type of dynamic throttling I envisioned in Camel. Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2047) camel-jms async consumer needs
dynamic mechanism to throttle message consumption
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=54563#action_54563 ]
Claus Ibsen commented on CAMEL-2047:
------------------------------------
If everyone goes inventing their own solutions to this, why haven't Spring provided this directly in their DMLC (spring-jms)?
Ron a good idea, however this does impact the consumer endpoint more than at first sight as it needs to be able to dynamic start/stop the DMLC. And this it needs a background thread running for that. Which of course is doable.
We also have a ticket for adding some sort of logic to a route consumer whether it should be running or not, so you can enable/disable routes dynamic at runtime.
There is a JIRA already for this.
But your idea goes one step further as its more tight integrated with the JMS as you need the dynamic throttler perspective.
> camel-jms async consumer needs dynamic mechanism to throttle message consumption
> --------------------------------------------------------------------------------
>
> Key: CAMEL-2047
> URL: https://issues.apache.org/activemq/browse/CAMEL-2047
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-jms
> Affects Versions: 2.0.0
> Reporter: Ron Gavlin
>
> camel-jms should provide an async consumer that can throttle consumption of incoming messages. This is necessary in scenarios where a large backlog of messages exists on the input queue. In this case, it is likely that an async consumer will overload camel with message exchanges resulting in problems. A dynamic throttling mechanism on the async camel-jms consumer is required to avoid this problem.
> One possible implementation might be as follows:
> First, add a property "maxPendingAsyncExchanges" on the async camel-jms consumer endpoint. Then add support in the endpoint to track the total number of pending/in-progress exchanges, i.e., the number of exchanges for which no SUCCESS/ERROR acknowledgement has been returned. When this value reaches "maxPendingAsyncExchanges", then the endpoint would stop its Spring JMS DefaultMessageListenerContainer. Once the number of "pending" exchanges drops below a percentage of the "maxPendingAsyncExchanges" threshold, say "reconnectPercentOfMax", the Spring DMLC would be restarted.
> The CXF implementation of this capability is described in CXF-2002, CXF-2342, and CXF-2391. CXF has properties maxSuspendedContinuations and reconnectPercentOfMax to control the throttling behavior.
> This is the type of dynamic throttling I envisioned in Camel. Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.