You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Stephen Gargan (JIRA)" <ji...@apache.org> on 2009/11/20 03:57:54 UTC
[jira] Created: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Addition of 'Sample' Throttling strategy
----------------------------------------
Key: CAMEL-2206
URL: https://issues.apache.org/activemq/browse/CAMEL-2206
Project: Apache Camel
Issue Type: Improvement
Components: camel-core, camel-spring
Affects Versions: 2.1.0
Reporter: Stephen Gargan
Priority: Minor
I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
This would be defined via the java dsl as follows
{code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
or
{code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
or with a default of one per second
{code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
In spring xml it looks like
{code}
<route>
<from uri="direct:sample" />
<sample>
<to uri="mock:result" />
</sample>
</route>
{code}
or
{code}
<route>
<from uri="direct:sample" />
<sample samplePeriod="1" units="seconds">
<to uri="mock:result" />
</sample>
</route>
{code}
If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Stephen Gargan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Gargan updated CAMEL-2206:
----------------------------------
Attachment: sampling-patch-spring.txt
sampling-patch-core.txt
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core, camel-spring
> Affects Versions: 2.1.0
> Reporter: Stephen Gargan
> Priority: Minor
> Attachments: sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Stephen Gargan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56033#action_56033 ]
Stephen Gargan commented on CAMEL-2206:
---------------------------------------
Claus,
I've made the changed you suggested. It may be marginally more efficient to calculate the stats by maintaining only the total number of samples and the total no of exchanges as the number dropped will always be the total - the no. sampled. I guess if you're running in TRACE mode though the question of efficiency is moot so for clarity I've used accumulators for each.
If you feel the need to change this or any other changes you see fit, then by all means make them, I do not mind ;)
I'll get back to you with the ICLA
Thanks,
ste
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Reporter: Stephen Gargan
> Fix For: 2.2.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core-with-stats.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55661#action_55661 ]
Claus Ibsen commented on CAMEL-2206:
------------------------------------
I wonder if the DEBUG logging for skipped Exchange should be TRACE level.
Then it wont spam logs for high throughputs and you run with DEBUG logging.
And the Exchange that is picked up as sample you could log how many was skipped in that period
For example the log could be a DEBUG level something like:
Sampling exchange out of XXX in the last XXX units
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Affects Versions: 2.1.0
> Reporter: Stephen Gargan
> Priority: Minor
> Fix For: 2.1.0, 2.2.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-2206:
-------------------------------
Priority: Major (was: Minor)
Affects Version/s: (was: 2.1.0)
Fix Version/s: (was: 2.1.0)
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Reporter: Stephen Gargan
> Fix For: 2.2.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Stephen Gargan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Gargan updated CAMEL-2206:
----------------------------------
Attachment: sampling-patch-core-with-stats.txt
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Reporter: Stephen Gargan
> Fix For: 2.2.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core-with-stats.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-2206.
--------------------------------
Resolution: Fixed
Fix Version/s: (was: 2.2.0)
2.1.0
trunk: 883614.
Thanks a lot for the contribution Stephen.
I have committed it and create a wiki page
http://cwiki.apache.org/confluence/display/CAMEL/Sampling
When you have the ICLA in place you will be able to edit the documentations as well.
In the mean time feel free to comment/feedback the current wiki page.
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Reporter: Stephen Gargan
> Assignee: Claus Ibsen
> Fix For: 2.1.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core-with-stats.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Stephen Gargan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Gargan updated CAMEL-2206:
----------------------------------
Attachment: sampling-patch-core-with-javadoc.txt
Whoops sorry about that. I've added the Adapter and updated the javadoc. If you give me write access for the path http://camel.apache.org/sampling.html write up the doc for it.
Nice one,
Stephen.
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Affects Versions: 2.1.0
> Reporter: Stephen Gargan
> Priority: Minor
> Fix For: 2.1.0, 2.2.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-2206:
-------------------------------
Fix Version/s: 2.2.0
2.1.0
Issue Type: New Feature (was: Improvement)
Lets see if we can get it into 2.1
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Affects Versions: 2.1.0
> Reporter: Stephen Gargan
> Priority: Minor
> Fix For: 2.1.0, 2.2.0
>
> Attachments: sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-2206:
----------------------------------
Assignee: Claus Ibsen
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Reporter: Stephen Gargan
> Assignee: Claus Ibsen
> Fix For: 2.2.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core-with-stats.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Stephen Gargan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stephen Gargan updated CAMEL-2206:
----------------------------------
Attachment: sampling-patch-core.txt
Complete core patch this time.
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Affects Versions: 2.1.0
> Reporter: Stephen Gargan
> Priority: Minor
> Fix For: 2.1.0, 2.2.0
>
> Attachments: sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55659#action_55659 ]
Claus Ibsen commented on CAMEL-2206:
------------------------------------
Stephen to edit the wiki pages you need to sign a ICLA
Read more here
http://camel.apache.org/how-do-i-edit-the-website.html
If you dont want to go that route you can create some form of documentation of the sampler and attach it to this JIRA and then I can create and edit the wiki page.
However by having the ICLA signed you are one step closer to contribute more and becoming a committer.
http://camel.apache.org/contributing.html
http://camel.apache.org/how-do-i-become-a-committer.html
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: New Feature
> Components: camel-core, camel-spring
> Affects Versions: 2.1.0
> Reporter: Stephen Gargan
> Priority: Minor
> Fix For: 2.1.0, 2.2.0
>
> Attachments: sampling-patch-core-with-javadoc.txt, sampling-patch-core.txt, sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-2206) Addition of 'Sample' Throttling
strategy
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=55623#action_55623 ]
Claus Ibsen commented on CAMEL-2206:
------------------------------------
Stephen looks really great.
I cannot find the class
import org.apache.camel.builder.xml.TimeUnitAdapter;
in the patch. Can you attach that file also?
We would definitely like to add this to the Camel. Thanks for contributing.
> Addition of 'Sample' Throttling strategy
> ----------------------------------------
>
> Key: CAMEL-2206
> URL: https://issues.apache.org/activemq/browse/CAMEL-2206
> Project: Apache Camel
> Issue Type: Improvement
> Components: camel-core, camel-spring
> Affects Versions: 2.1.0
> Reporter: Stephen Gargan
> Priority: Minor
> Attachments: sampling-patch-core.txt, sampling-patch-spring.txt
>
>
> I've been using a different kind of sampling based throttling pattern recently and figured others might find it useful. The basic premise is the same as the current throttler, there is a consumer downstream that has specific requirements as to how fast it can process incoming exchanges and a mechanism is required to 'throttle' the exchanges inbound to it. The difference between the current throttler implementation and the sampling throttler is that where the current throttler queues up all the exchanges it receives, the sampling throttler will allow only a single exchange through for a period and 'drop' all others; dropped exchanges being passed to a stop processor to complete them.
> Its been useful for situations where a route receives many semantically equivalent exchanges in a time period, any single one of which could be used to represent them all. For example, say an upstream component generates a stream of warning alarms and sends these to an email processor to inform the operator. An email for each alarm would be overkill instead one every x minutes or so would be preferred. Also it would be undesirable for the throttler to queue up these warning alarms as when the upstream component ceases sending them the emails should cease. Sampling the warnings would pick one for each x minute period, and stop the rest.
> This would be defined via the java dsl as follows
> {code}from("direct:alarm-notifier").sample(60, TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or
> {code}from("direct:alarm-notifier").sample().samplePeriod(1).timeUnits(TimeUnit.SECONDS).to("direct:alarm-emailer"){code}
> or with a default of one per second
> {code}from("direct:alarm-notifier").sample().to("direct:alarm-emailer"){code}
> In spring xml it looks like
> {code}
> <route>
> <from uri="direct:sample" />
> <sample>
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> or
> {code}
> <route>
> <from uri="direct:sample" />
> <sample samplePeriod="1" units="seconds">
> <to uri="mock:result" />
> </sample>
> </route>
> {code}
> If this patch is accepted, I'd be happy to update the wiki with the required documentation. As per ususal I've split the patches, one for core and one for spring
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.