Asynchronous Routing Engine - Status

[Claus Ibsen <cl...@gmail.com>]

Hi

The major goal for Camel 2.4 is introducing back the asynchronous
routing engine from the Camel 1.x days.
What we had replaced it with, just didn't cut it for the long run, so
we had to toss it out and replace it back with the 1.x variation.
However the 1.x variation is much more invasive in the camel core.
However the benefits are worth it.

There is a JIRA ticket to track this
https://issues.apache.org/activemq/browse/CAMEL-2723

We got so far with the code that we can move ahead the release
schedule for 2.4 and thus why Hadrian send out
an announcement yesterday.

The engine has been integrated with Apache ServiceMix JBI and NMR,
which means that in SMX 4.3 and Came. 2.4 onwards
you have much better scalability and utilization of the resources.

There is present only a single Camel component which leverages the
async routing as well which is Jetty.
The consumer side uses Jetty continuations and the producer side the
Jetty http async client.

We plan to expand this to other Camel components where async
communication is native supported by the component.
Such as Netty, Mina 2 (when its out), CXF producer, REST stuff and what else.

Mind that async routing does not work to the fullest if you are not
using transactions. Spring TM requires the transactional resources to
be
executed from the same thread context, which is why Camel will
fallback to route fully synchronous if you use transacted routes.
This works automatic and you dont have to worry about it to configure
any of the Camel endpoints to cater for this. It's just that Camel
will have the caller thread blocked while waiting for the async reply,
in case you for example to request/reply over NMR in a transaction.


Almost all the EIP patterns in Camel have been refactored to support
async routing engine as well. So you can use Splitter, Recipient List,
WireTap
and all the others in the most optimal way. Currently there are 3
outstanding works to be looked at:
- Throttler and Delayer EIP to not block while waiting
- Multicast, Recipient List and Splitter EIP to support async routing
engine when parallelProcessing option is enabled (this is a bit tricky
to do).
- RedeliverErrorHandler to use non blocking while waiting for delivery
(current only enabled if async routing has been kicked in) but we can
do this in a general.




-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: Asynchronous Routing Engine - Status

[Hiram Chirino <hi...@hiramchirino.com>]

Awesome work.  This should really provide a huge scalability boost for some
folks!

On Thu, Jul 1, 2010 at 7:58 AM, Claus Ibsen <cl...@gmail.com> wrote:

> Hi
>
> The major goal for Camel 2.4 is introducing back the asynchronous
> routing engine from the Camel 1.x days.
> What we had replaced it with, just didn't cut it for the long run, so
> we had to toss it out and replace it back with the 1.x variation.
> However the 1.x variation is much more invasive in the camel core.
> However the benefits are worth it.
>
> There is a JIRA ticket to track this
> https://issues.apache.org/activemq/browse/CAMEL-2723
>
> We got so far with the code that we can move ahead the release
> schedule for 2.4 and thus why Hadrian send out
> an announcement yesterday.
>
> The engine has been integrated with Apache ServiceMix JBI and NMR,
> which means that in SMX 4.3 and Came. 2.4 onwards
> you have much better scalability and utilization of the resources.
>
> There is present only a single Camel component which leverages the
> async routing as well which is Jetty.
> The consumer side uses Jetty continuations and the producer side the
> Jetty http async client.
>
> We plan to expand this to other Camel components where async
> communication is native supported by the component.
> Such as Netty, Mina 2 (when its out), CXF producer, REST stuff and what
> else.
>
> Mind that async routing does not work to the fullest if you are not
> using transactions. Spring TM requires the transactional resources to
> be
> executed from the same thread context, which is why Camel will
> fallback to route fully synchronous if you use transacted routes.
> This works automatic and you dont have to worry about it to configure
> any of the Camel endpoints to cater for this. It's just that Camel
> will have the caller thread blocked while waiting for the async reply,
> in case you for example to request/reply over NMR in a transaction.
>
>
> Almost all the EIP patterns in Camel have been refactored to support
> async routing engine as well. So you can use Splitter, Recipient List,
> WireTap
> and all the others in the most optimal way. Currently there are 3
> outstanding works to be looked at:
> - Throttler and Delayer EIP to not block while waiting
> - Multicast, Recipient List and Splitter EIP to support async routing
> engine when parallelProcessing option is enabled (this is a bit tricky
> to do).
> - RedeliverErrorHandler to use non blocking while waiting for delivery
> (current only enabled if async routing has been kicked in) but we can
> do this in a general.
>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Open Source SOA
http://fusesource.com/

Re: Asynchronous Routing Engine - Status

[Claus Ibsen <cl...@gmail.com>]

On Fri, Jul 2, 2010 at 3:33 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Just to update the status that today I have implemented
>
> a) error handler will now schedule delayed redeliveries by default
> which thus allows to not block while the delay occurs. You can disable
> this behavior using the syncDelayed() option on the error handler /
> onException / routePolicy. And if you use transactions it will use
> synced delays to ensure the exchange will use the same thread again as
> the Transaction Manager requires to execute in the same thread
> context.
>

Just a though.

Do anyone think it would be best to keep the old behavior as default?
To block while delaying when routing synchronous?

For a single consumer thread this could cause it to be able to pickup
new messages which might also fail
and thus stack up messages on the delayed redelivery task queue?

I think I will flip back the default so it works as it have always
done, then people can turn this option on to have async delayed if
they want.



> b) delay and throttle EIP now have an option asyncDelayed() you can
> use to enable them to use non blocking delay as well
>
> The only question though would be if we should have an option to allow
> you to cancel tasks on shutdown. If you have messages queued up in the
> delay/throttle. Then a fast stop to cancel the pending tasks is
> possible. However the JDK ExecutorService API is damn poor in terms of
> stopping the thread pool. You can only shut it down which then cant
> restart it. And getting the pending tasks is also not possible unless
> you gather the Future references and then remember to keep this list
> up to date removing the done tasks.
>
>
> On Thu, Jul 1, 2010 at 1:58 PM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> The major goal for Camel 2.4 is introducing back the asynchronous
>> routing engine from the Camel 1.x days.
>> What we had replaced it with, just didn't cut it for the long run, so
>> we had to toss it out and replace it back with the 1.x variation.
>> However the 1.x variation is much more invasive in the camel core.
>> However the benefits are worth it.
>>
>> There is a JIRA ticket to track this
>> https://issues.apache.org/activemq/browse/CAMEL-2723
>>
>> We got so far with the code that we can move ahead the release
>> schedule for 2.4 and thus why Hadrian send out
>> an announcement yesterday.
>>
>> The engine has been integrated with Apache ServiceMix JBI and NMR,
>> which means that in SMX 4.3 and Came. 2.4 onwards
>> you have much better scalability and utilization of the resources.
>>
>> There is present only a single Camel component which leverages the
>> async routing as well which is Jetty.
>> The consumer side uses Jetty continuations and the producer side the
>> Jetty http async client.
>>
>> We plan to expand this to other Camel components where async
>> communication is native supported by the component.
>> Such as Netty, Mina 2 (when its out), CXF producer, REST stuff and what else.
>>
>> Mind that async routing does not work to the fullest if you are not
>> using transactions. Spring TM requires the transactional resources to
>> be
>> executed from the same thread context, which is why Camel will
>> fallback to route fully synchronous if you use transacted routes.
>> This works automatic and you dont have to worry about it to configure
>> any of the Camel endpoints to cater for this. It's just that Camel
>> will have the caller thread blocked while waiting for the async reply,
>> in case you for example to request/reply over NMR in a transaction.
>>
>>
>> Almost all the EIP patterns in Camel have been refactored to support
>> async routing engine as well. So you can use Splitter, Recipient List,
>> WireTap
>> and all the others in the most optimal way. Currently there are 3
>> outstanding works to be looked at:
>> - Throttler and Delayer EIP to not block while waiting
>> - Multicast, Recipient List and Splitter EIP to support async routing
>> engine when parallelProcessing option is enabled (this is a bit tricky
>> to do).
>> - RedeliverErrorHandler to use non blocking while waiting for delivery
>> (current only enabled if async routing has been kicked in) but we can
>> do this in a general.
>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: Asynchronous Routing Engine - Status

[Claus Ibsen <cl...@gmail.com>]

Hi

Just to update the status that today I have implemented

a) error handler will now schedule delayed redeliveries by default
which thus allows to not block while the delay occurs. You can disable
this behavior using the syncDelayed() option on the error handler /
onException / routePolicy. And if you use transactions it will use
synced delays to ensure the exchange will use the same thread again as
the Transaction Manager requires to execute in the same thread
context.

b) delay and throttle EIP now have an option asyncDelayed() you can
use to enable them to use non blocking delay as well

The only question though would be if we should have an option to allow
you to cancel tasks on shutdown. If you have messages queued up in the
delay/throttle. Then a fast stop to cancel the pending tasks is
possible. However the JDK ExecutorService API is damn poor in terms of
stopping the thread pool. You can only shut it down which then cant
restart it. And getting the pending tasks is also not possible unless
you gather the Future references and then remember to keep this list
up to date removing the done tasks.


On Thu, Jul 1, 2010 at 1:58 PM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> The major goal for Camel 2.4 is introducing back the asynchronous
> routing engine from the Camel 1.x days.
> What we had replaced it with, just didn't cut it for the long run, so
> we had to toss it out and replace it back with the 1.x variation.
> However the 1.x variation is much more invasive in the camel core.
> However the benefits are worth it.
>
> There is a JIRA ticket to track this
> https://issues.apache.org/activemq/browse/CAMEL-2723
>
> We got so far with the code that we can move ahead the release
> schedule for 2.4 and thus why Hadrian send out
> an announcement yesterday.
>
> The engine has been integrated with Apache ServiceMix JBI and NMR,
> which means that in SMX 4.3 and Came. 2.4 onwards
> you have much better scalability and utilization of the resources.
>
> There is present only a single Camel component which leverages the
> async routing as well which is Jetty.
> The consumer side uses Jetty continuations and the producer side the
> Jetty http async client.
>
> We plan to expand this to other Camel components where async
> communication is native supported by the component.
> Such as Netty, Mina 2 (when its out), CXF producer, REST stuff and what else.
>
> Mind that async routing does not work to the fullest if you are not
> using transactions. Spring TM requires the transactional resources to
> be
> executed from the same thread context, which is why Camel will
> fallback to route fully synchronous if you use transacted routes.
> This works automatic and you dont have to worry about it to configure
> any of the Camel endpoints to cater for this. It's just that Camel
> will have the caller thread blocked while waiting for the async reply,
> in case you for example to request/reply over NMR in a transaction.
>
>
> Almost all the EIP patterns in Camel have been refactored to support
> async routing engine as well. So you can use Splitter, Recipient List,
> WireTap
> and all the others in the most optimal way. Currently there are 3
> outstanding works to be looked at:
> - Throttler and Delayer EIP to not block while waiting
> - Multicast, Recipient List and Splitter EIP to support async routing
> engine when parallelProcessing option is enabled (this is a bit tricky
> to do).
> - RedeliverErrorHandler to use non blocking while waiting for delivery
> (current only enabled if async routing has been kicked in) but we can
> do this in a general.
>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: Asynchronous Routing Engine - Status

[Claus Ibsen <cl...@gmail.com>]

Hi

camel-netty producer is now async as well.

However I wonder if NettyConsumer should be async as well?

We do have the Channel in which a reply should be send back to, which
allows us to use it from the async callback.
Which should allow the NettyConsumer thread to return and let Netty
reuse that thread to service new requests.

I am just wondering if keeping that Channel around longer than the
original request thread would cause any issues?

I was looking for continuations support in Netty but there is no such
thing currently. This would obviously make it a first class feature in
Netty
and make me more comfortable to let the continuations handle returning
the reply when the async routing is done.




-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: Asynchronous Routing Engine - Status

[Claus Ibsen <cl...@gmail.com>]

On Fri, Jul 2, 2010 at 4:34 PM, Guillaume Nodet <gn...@gmail.com> wrote:
> Great work, Claus!
>
> On Thu, Jul 1, 2010 at 13:58, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> The major goal for Camel 2.4 is introducing back the asynchronous
>> routing engine from the Camel 1.x days.
>> What we had replaced it with, just didn't cut it for the long run, so
>> we had to toss it out and replace it back with the 1.x variation.
>> However the 1.x variation is much more invasive in the camel core.
>> However the benefits are worth it.
>>
>> There is a JIRA ticket to track this
>> https://issues.apache.org/activemq/browse/CAMEL-2723
>>
>> We got so far with the code that we can move ahead the release
>> schedule for 2.4 and thus why Hadrian send out
>> an announcement yesterday.
>>
>> The engine has been integrated with Apache ServiceMix JBI and NMR,
>> which means that in SMX 4.3 and Came. 2.4 onwards
>> you have much better scalability and utilization of the resources.
>>
>> There is present only a single Camel component which leverages the
>> async routing as well which is Jetty.
>> The consumer side uses Jetty continuations and the producer side the
>> Jetty http async client.
>>
>> We plan to expand this to other Camel components where async
>> communication is native supported by the component.
>> Such as Netty, Mina 2 (when its out), CXF producer, REST stuff and what else.
>>
>> Mind that async routing does not work to the fullest if you are not
>> using transactions. Spring TM requires the transactional resources to
>> be
>> executed from the same thread context, which is why Camel will
>> fallback to route fully synchronous if you use transacted routes.
>> This works automatic and you dont have to worry about it to configure
>> any of the Camel endpoints to cater for this. It's just that Camel
>> will have the caller thread blocked while waiting for the async reply,
>> in case you for example to request/reply over NMR in a transaction.
>>
>>
>
> We might be able to alleviate this limitation. I know the spring
> transaction layer can suspend and resume transactions in different
> threads.
> I've done this kind of integration already: see the
> registerTransactionAssociationListener method in the following class
>  https://svn.apache.org/repos/asf/incubator/aries/trunk/transaction/transaction-manager/src/main/java/org/apache/aries/transaction/GeronimoPlatformTransactionManager.java
>

Fantastic this looks great. I have created a ticket to track it
https://issues.apache.org/activemq/browse/CAMEL-2902

Also there is another ticket with a related feature for the Spring
DMLC to handle the JMS ACKs yourself.
https://issues.apache.org/activemq/browse/CAMEL-2729


So I think in Camel 2.5 we should be able to work on supporting async
TX as well at some level.



>> Almost all the EIP patterns in Camel have been refactored to support
>> async routing engine as well. So you can use Splitter, Recipient List,
>> WireTap
>> and all the others in the most optimal way. Currently there are 3
>> outstanding works to be looked at:
>> - Throttler and Delayer EIP to not block while waiting
>> - Multicast, Recipient List and Splitter EIP to support async routing
>> engine when parallelProcessing option is enabled (this is a bit tricky
>> to do).
>> - RedeliverErrorHandler to use non blocking while waiting for delivery
>> (current only enabled if async routing has been kicked in) but we can
>> do this in a general.
>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>
>
> --
> Cheers,
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
> ------------------------
> Open Source SOA
> http://fusesource.com
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: Asynchronous Routing Engine - Status

[Guillaume Nodet <gn...@gmail.com>]

Great work, Claus!

On Thu, Jul 1, 2010 at 13:58, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> The major goal for Camel 2.4 is introducing back the asynchronous
> routing engine from the Camel 1.x days.
> What we had replaced it with, just didn't cut it for the long run, so
> we had to toss it out and replace it back with the 1.x variation.
> However the 1.x variation is much more invasive in the camel core.
> However the benefits are worth it.
>
> There is a JIRA ticket to track this
> https://issues.apache.org/activemq/browse/CAMEL-2723
>
> We got so far with the code that we can move ahead the release
> schedule for 2.4 and thus why Hadrian send out
> an announcement yesterday.
>
> The engine has been integrated with Apache ServiceMix JBI and NMR,
> which means that in SMX 4.3 and Came. 2.4 onwards
> you have much better scalability and utilization of the resources.
>
> There is present only a single Camel component which leverages the
> async routing as well which is Jetty.
> The consumer side uses Jetty continuations and the producer side the
> Jetty http async client.
>
> We plan to expand this to other Camel components where async
> communication is native supported by the component.
> Such as Netty, Mina 2 (when its out), CXF producer, REST stuff and what else.
>
> Mind that async routing does not work to the fullest if you are not
> using transactions. Spring TM requires the transactional resources to
> be
> executed from the same thread context, which is why Camel will
> fallback to route fully synchronous if you use transacted routes.
> This works automatic and you dont have to worry about it to configure
> any of the Camel endpoints to cater for this. It's just that Camel
> will have the caller thread blocked while waiting for the async reply,
> in case you for example to request/reply over NMR in a transaction.
>
>

We might be able to alleviate this limitation. I know the spring
transaction layer can suspend and resume transactions in different
threads.
I've done this kind of integration already: see the
registerTransactionAssociationListener method in the following class
  https://svn.apache.org/repos/asf/incubator/aries/trunk/transaction/transaction-manager/src/main/java/org/apache/aries/transaction/GeronimoPlatformTransactionManager.java

> Almost all the EIP patterns in Camel have been refactored to support
> async routing engine as well. So you can use Splitter, Recipient List,
> WireTap
> and all the others in the most optimal way. Currently there are 3
> outstanding works to be looked at:
> - Throttler and Delayer EIP to not block while waiting
> - Multicast, Recipient List and Splitter EIP to support async routing
> engine when parallelProcessing option is enabled (this is a bit tricky
> to do).
> - RedeliverErrorHandler to use non blocking while waiting for delivery
> (current only enabled if async routing has been kicked in) but we can
> do this in a general.
>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com