Exception based routing

[Neil Thorne <mr...@hotmail.com>]

Hi,

I work with Nick Outram - we've a splitter of sorts working (with our own
processor)

I have the requirement where on a route from A to B there are a couple of
different exceptions that can be thrown.

a) a system exception gets thrown. A system exception indicates an
infrastructure problem that will be fixed ASAP as it is essential to the
correct functioning of the system as a whole. eg. jdbc connection failed,
remote system is down etc. If a system exception is thrown then that
operation should be continually retried until it is successful.

b) a business exception is thrown. A business exception indicates that there
was a problem with the request itself. It should not be retried (at least
not until the problem has been resolved). The message should be moved onto a
hospital queue where it can be inspected, changed and retried after possible
manual intervention.

What is the simplest way to achieve this with camel? Currently I can only
see an errorHandler which assumes you want to dump messages that cannot be
processed to an error endpoint. Can I bind an ErrorHandler to a specific
type of exception on the same route somehow?

If I could then I suppose I could move business exceptions immediately to
the hospital queue (with a retry count of zero).

I could also bind a different error handler with an infinite retry count
(-1?).

thanks,

Neil
-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12024679
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/16/07, Nick Outram <ap...@nickoutram.name> wrote:
> James.Strachan wrote:
> > You can see this DSL in action using a variation of the ValidationTest...
> > https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ValidationWithExceptionTest.java
> >
> > I'll ponder how easy its gonna be to use the DSL to customize
> > redelivery policy stuff on a per type basis too...
>
> For a bean: endpoint the exception handling fails as all pojo exceptions are
> converted to InvocationTargetException...
>
> The exception needs to be unwrapped, not sure where's best though;
> MethodInfo.invoke()? or maybe ErrorHandlerSupport.getExceptionPolicy()?

Great catch! Yeah I think the MethodInfo.invoke() (BeanProcessor)
should be the place (i.e. any code that does reflection to call a bean
should unwrap this exception).

I raised a JIRA to track this one...
https://issues.apache.org/activemq/browse/CAMEL-111

I've just committed a fix for this along with a test case showing it working..
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/BeanWithExceptionTest.java

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Nick Outram <ap...@nickoutram.name>]

James.Strachan wrote:
> 
> 
> You can see this DSL in action using a variation of the ValidationTest...
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ValidationWithExceptionTest.java
> 
> I'll ponder how easy its gonna be to use the DSL to customize
> redelivery policy stuff on a per type basis too...
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> 

For a bean: endpoint the exception handling fails as all pojo exceptions are
converted to InvocationTargetException...

The exception needs to be unwrapped, not sure where's best though;
MethodInfo.invoke()? or maybe ErrorHandlerSupport.getExceptionPolicy()?

-N
-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12183747
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/10/07, Hiram Chirino <hi...@hiramchirino.com> wrote:
> On 8/10/07, James Strachan <ja...@gmail.com> wrote:
> > On 8/10/07, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > > On 8/10/07, James Strachan <ja...@gmail.com> wrote:
> > > > Here's a thought. The default error handler used in Camel is the
> > > > DeadLetterChannel; in an attempt to do what most people want out of
> > > > the box without having to explicitly customize things.
> > > >
> > > > So most errors tend to be transient system issues (e.g. database
> > > > deadlocks, temporary unavailability of connections/dependent
> > > > services), so retrying N times before sending to a dead letter channel
> > > > is a reasonable default.
> > > >
> > > > When a user customizes certain exceptions in a route; its often gonna
> > > > be a business exception or something similar. e.g. validation
> > > > exceptions (whether XSD or business logic constraint violations etc).
> > > > Usually for these cases, you don't want redelivery at all - since the
> > > > exception isn't gonna change as its business logic.
> > > >
> > > > So I was thinking, the default case should be that specifying an
> > > > exception handler route - without doing anything else - would
> > > > effectively by default, turn off retry for that exception, unless the
> > > > user explicitly customizes the retry count for that exception - or
> > > > maybe the user customizes the default value for all exception()
> > > > policies.
> > > >
> > > > e.g.
> > > >
> > > > exception(Validation.class).to("seda:badlyValidatedMessages");
> > > > from("activemq:foo").to("jpa:MyEntityBean");
> > > >
> > > > if we get a database exception, we retry N times. However by default
> > > > if we get a validation error we don't retry and just immediately send
> > > > to "seda:badlyValidatedMessages".
> > > >
> > > > If you don't want this, you can still be explicit...
> > > >
> > > > // lets retry for this exception...
> > > > exception(NullPointerException.class).maximumRetries(3);
> > > >
> > > > // otherwise lets not retry for validations...
> > > > exception(Validation.class).to("seda:badlyValidatedMessages");
> > > >
> > > > from("activemq:foo").to("jpa:MyEntityBean");
> > > >
> > > > Does that seem reasonable & non-confusing? Am a tad conscious of
> > > > adding too much magic under the covers - but in many ways conceptually
> > > > I kinda think when someone uses the code
> > > >
> > > >   exception(that).to(somewhere);
> > > >
> > > > they are kinda saying 'on that exception, go to somewhere" - which
> > > > kinda implies there'd be no retry logic applied by default - unless
> > > > you explicitly say you want that.
> > > >
> > >
> > > This sounds good to me..
> >
> > Cool!
> >
> >
> > > And a question I have.. will it be possible in 1 route builder to
> > > define a global exception handler for one set of routes and then
> > > change it to different one for a different set of routes.
> > >
> > > For example, if we want to keep each customer's messages on different
> > > jms queues:
> > >
> > >  exception(Validation.class).to("activemq:cust-a-failures");
> > >  from("activemq:cust-a-po").to("seda:po");
> > >  from("activemq:cust-a-invoice").to("seda:invoice");
> > >
> > >  exception(Validation.class).to("activemq:cust-b-failures");
> > >  from("activemq:cust-b-po").to("seda:po");
> > >  from("activemq:cust-b-invoice").to("seda:nvoice");
> >
> > Yeah - currently each exception(Foo.class) will override the previous
> > one (currently only one exception policy/handler is allowed per
> > exception type).
> >
>
> That's too bad.. Would you at least be able to do the above with
> multiple route builders on a single context?

Oh definitely - the previous restrictions I mention are purely down to
a single RouteBuilder instance. We could add some clear kinda method
to zap any interceptors/error handlers in between defining groups of
routes maybe?


-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Hiram Chirino <hi...@hiramchirino.com>]

On 8/10/07, James Strachan <ja...@gmail.com> wrote:
> On 8/10/07, Hiram Chirino <hi...@hiramchirino.com> wrote:
> > On 8/10/07, James Strachan <ja...@gmail.com> wrote:
> > > Here's a thought. The default error handler used in Camel is the
> > > DeadLetterChannel; in an attempt to do what most people want out of
> > > the box without having to explicitly customize things.
> > >
> > > So most errors tend to be transient system issues (e.g. database
> > > deadlocks, temporary unavailability of connections/dependent
> > > services), so retrying N times before sending to a dead letter channel
> > > is a reasonable default.
> > >
> > > When a user customizes certain exceptions in a route; its often gonna
> > > be a business exception or something similar. e.g. validation
> > > exceptions (whether XSD or business logic constraint violations etc).
> > > Usually for these cases, you don't want redelivery at all - since the
> > > exception isn't gonna change as its business logic.
> > >
> > > So I was thinking, the default case should be that specifying an
> > > exception handler route - without doing anything else - would
> > > effectively by default, turn off retry for that exception, unless the
> > > user explicitly customizes the retry count for that exception - or
> > > maybe the user customizes the default value for all exception()
> > > policies.
> > >
> > > e.g.
> > >
> > > exception(Validation.class).to("seda:badlyValidatedMessages");
> > > from("activemq:foo").to("jpa:MyEntityBean");
> > >
> > > if we get a database exception, we retry N times. However by default
> > > if we get a validation error we don't retry and just immediately send
> > > to "seda:badlyValidatedMessages".
> > >
> > > If you don't want this, you can still be explicit...
> > >
> > > // lets retry for this exception...
> > > exception(NullPointerException.class).maximumRetries(3);
> > >
> > > // otherwise lets not retry for validations...
> > > exception(Validation.class).to("seda:badlyValidatedMessages");
> > >
> > > from("activemq:foo").to("jpa:MyEntityBean");
> > >
> > > Does that seem reasonable & non-confusing? Am a tad conscious of
> > > adding too much magic under the covers - but in many ways conceptually
> > > I kinda think when someone uses the code
> > >
> > >   exception(that).to(somewhere);
> > >
> > > they are kinda saying 'on that exception, go to somewhere" - which
> > > kinda implies there'd be no retry logic applied by default - unless
> > > you explicitly say you want that.
> > >
> >
> > This sounds good to me..
>
> Cool!
>
>
> > And a question I have.. will it be possible in 1 route builder to
> > define a global exception handler for one set of routes and then
> > change it to different one for a different set of routes.
> >
> > For example, if we want to keep each customer's messages on different
> > jms queues:
> >
> >  exception(Validation.class).to("activemq:cust-a-failures");
> >  from("activemq:cust-a-po").to("seda:po");
> >  from("activemq:cust-a-invoice").to("seda:invoice");
> >
> >  exception(Validation.class).to("activemq:cust-b-failures");
> >  from("activemq:cust-b-po").to("seda:po");
> >  from("activemq:cust-b-invoice").to("seda:nvoice");
>
> Yeah - currently each exception(Foo.class) will override the previous
> one (currently only one exception policy/handler is allowed per
> exception type).
>

That's too bad.. Would you at least be able to do the above with
multiple route builders on a single context?

> The exception to this (pun intended :) is for interceptors - if using
> intercept(), they are always added for each intercept() call; so we'd
> need some kinda clearInterceptors() or something if you want to remove
> previous interceptors; in that case it might be cleaner to have
> separate builder for rules which don't have a previously registered
> interceptor applied.
>
> Or some nice DSL way to disable certain interceptors; or to include a
> predicate in the interceptor which allows you to excluse the routes
> you don't want to use it on :)
>



> --
> James
> -------
> http://macstrac.blogspot.com/
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

All of Camel should work inside ServiceMix; the only thing to be
careful of is making sure you've a recent enough ServiceMix distro
thats using a recent Camel release in servicemix-camel.

2008/5/20 pratibhaG <pr...@in2m.com>:
>
> All these are great solutions.
> I am working on similar kind of problem as Neil is working on as he had
> explained in the first post.
> I am using camel inside servicemix version 3.2.1. That is I am using
> servicemix-camel component provided by servicemix. Could anybody tell me if
> I can use all these features if I use camel inside servicemix and not a
> standalone camel?
>
> Regards,
> Pratibha
> --
> View this message in context: http://www.nabble.com/Exception-based-routing-tp12024679s22882p17337652.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Re: Exception based routing

[pratibhaG <pr...@in2m.com>]

All these are great solutions.
I am working on similar kind of problem as Neil is working on as he had
explained in the first post.
I am using camel inside servicemix version 3.2.1. That is I am using
servicemix-camel component provided by servicemix. Could anybody tell me if
I can use all these features if I use camel inside servicemix and not a
standalone camel?

Regards,
Pratibha
-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tp12024679s22882p17337652.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/10/07, Hiram Chirino <hi...@hiramchirino.com> wrote:
> On 8/10/07, James Strachan <ja...@gmail.com> wrote:
> > Here's a thought. The default error handler used in Camel is the
> > DeadLetterChannel; in an attempt to do what most people want out of
> > the box without having to explicitly customize things.
> >
> > So most errors tend to be transient system issues (e.g. database
> > deadlocks, temporary unavailability of connections/dependent
> > services), so retrying N times before sending to a dead letter channel
> > is a reasonable default.
> >
> > When a user customizes certain exceptions in a route; its often gonna
> > be a business exception or something similar. e.g. validation
> > exceptions (whether XSD or business logic constraint violations etc).
> > Usually for these cases, you don't want redelivery at all - since the
> > exception isn't gonna change as its business logic.
> >
> > So I was thinking, the default case should be that specifying an
> > exception handler route - without doing anything else - would
> > effectively by default, turn off retry for that exception, unless the
> > user explicitly customizes the retry count for that exception - or
> > maybe the user customizes the default value for all exception()
> > policies.
> >
> > e.g.
> >
> > exception(Validation.class).to("seda:badlyValidatedMessages");
> > from("activemq:foo").to("jpa:MyEntityBean");
> >
> > if we get a database exception, we retry N times. However by default
> > if we get a validation error we don't retry and just immediately send
> > to "seda:badlyValidatedMessages".
> >
> > If you don't want this, you can still be explicit...
> >
> > // lets retry for this exception...
> > exception(NullPointerException.class).maximumRetries(3);
> >
> > // otherwise lets not retry for validations...
> > exception(Validation.class).to("seda:badlyValidatedMessages");
> >
> > from("activemq:foo").to("jpa:MyEntityBean");
> >
> > Does that seem reasonable & non-confusing? Am a tad conscious of
> > adding too much magic under the covers - but in many ways conceptually
> > I kinda think when someone uses the code
> >
> >   exception(that).to(somewhere);
> >
> > they are kinda saying 'on that exception, go to somewhere" - which
> > kinda implies there'd be no retry logic applied by default - unless
> > you explicitly say you want that.
> >
>
> This sounds good to me..

Cool!


> And a question I have.. will it be possible in 1 route builder to
> define a global exception handler for one set of routes and then
> change it to different one for a different set of routes.
>
> For example, if we want to keep each customer's messages on different
> jms queues:
>
>  exception(Validation.class).to("activemq:cust-a-failures");
>  from("activemq:cust-a-po").to("seda:po");
>  from("activemq:cust-a-invoice").to("seda:invoice");
>
>  exception(Validation.class).to("activemq:cust-b-failures");
>  from("activemq:cust-b-po").to("seda:po");
>  from("activemq:cust-b-invoice").to("seda:nvoice");

Yeah - currently each exception(Foo.class) will override the previous
one (currently only one exception policy/handler is allowed per
exception type).

The exception to this (pun intended :) is for interceptors - if using
intercept(), they are always added for each intercept() call; so we'd
need some kinda clearInterceptors() or something if you want to remove
previous interceptors; in that case it might be cleaner to have
separate builder for rules which don't have a previously registered
interceptor applied.

Or some nice DSL way to disable certain interceptors; or to include a
predicate in the interceptor which allows you to excluse the routes
you don't want to use it on :)

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Hiram Chirino <hi...@hiramchirino.com>]

On 8/10/07, James Strachan <ja...@gmail.com> wrote:
> Here's a thought. The default error handler used in Camel is the
> DeadLetterChannel; in an attempt to do what most people want out of
> the box without having to explicitly customize things.
>
> So most errors tend to be transient system issues (e.g. database
> deadlocks, temporary unavailability of connections/dependent
> services), so retrying N times before sending to a dead letter channel
> is a reasonable default.
>
> When a user customizes certain exceptions in a route; its often gonna
> be a business exception or something similar. e.g. validation
> exceptions (whether XSD or business logic constraint violations etc).
> Usually for these cases, you don't want redelivery at all - since the
> exception isn't gonna change as its business logic.
>
> So I was thinking, the default case should be that specifying an
> exception handler route - without doing anything else - would
> effectively by default, turn off retry for that exception, unless the
> user explicitly customizes the retry count for that exception - or
> maybe the user customizes the default value for all exception()
> policies.
>
> e.g.
>
> exception(Validation.class).to("seda:badlyValidatedMessages");
> from("activemq:foo").to("jpa:MyEntityBean");
>
> if we get a database exception, we retry N times. However by default
> if we get a validation error we don't retry and just immediately send
> to "seda:badlyValidatedMessages".
>
> If you don't want this, you can still be explicit...
>
> // lets retry for this exception...
> exception(NullPointerException.class).maximumRetries(3);
>
> // otherwise lets not retry for validations...
> exception(Validation.class).to("seda:badlyValidatedMessages");
>
> from("activemq:foo").to("jpa:MyEntityBean");
>
> Does that seem reasonable & non-confusing? Am a tad conscious of
> adding too much magic under the covers - but in many ways conceptually
> I kinda think when someone uses the code
>
>   exception(that).to(somewhere);
>
> they are kinda saying 'on that exception, go to somewhere" - which
> kinda implies there'd be no retry logic applied by default - unless
> you explicitly say you want that.
>

This sounds good to me..
And a question I have.. will it be possible in 1 route builder to
define a global exception handler for one set of routes and then
change it to different one for a different set of routes.

For example, if we want to keep each customer's messages on different
jms queues:

 exception(Validation.class).to("activemq:cust-a-failures");
 from("activemq:cust-a-po").to("seda:po");
 from("activemq:cust-a-invoice").to("seda:invoice");

 exception(Validation.class).to("activemq:cust-b-failures");
 from("activemq:cust-b-po").to("seda:po");
 from("activemq:cust-b-invoice").to("seda:nvoice");



> --
> James
> -------
> http://macstrac.blogspot.com/
>


-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/14/07, Nick Outram <ap...@nickoutram.name> wrote:
> James.Strachan wrote:
> >
> > Here's a thought. The default error handler used in Camel is the
> > DeadLetterChannel; in an attempt to do what most people want out of
> > the box without having to explicitly customize things.
> >
> > So most errors tend to be transient system issues (e.g. database
> > deadlocks, temporary unavailability of connections/dependent
> > services), so retrying N times before sending to a dead letter channel
> > is a reasonable default.
> >
> > When a user customizes certain exceptions in a route; its often gonna
> > be a business exception or something similar. e.g. validation
> > exceptions (whether XSD or business logic constraint violations etc).
> > Usually for these cases, you don't want redelivery at all - since the
> > exception isn't gonna change as its business logic.
> >
> > So I was thinking, the default case should be that specifying an
> > exception handler route - without doing anything else - would
> > effectively by default, turn off retry for that exception, unless the
> > user explicitly customizes the retry count for that exception - or
> > maybe the user customizes the default value for all exception()
> > policies.
> >
> > e.g.
> >
> > exception(Validation.class).to("seda:badlyValidatedMessages");
> > from("activemq:foo").to("jpa:MyEntityBean");
> >
> > if we get a database exception, we retry N times. However by default
> > if we get a validation error we don't retry and just immediately send
> > to "seda:badlyValidatedMessages".
> >
> > If you don't want this, you can still be explicit...
> >
> > // lets retry for this exception...
> > exception(NullPointerException.class).maximumRetries(3);
> >
> > // otherwise lets not retry for validations...
> > exception(Validation.class).to("seda:badlyValidatedMessages");
> >
> > from("activemq:foo").to("jpa:MyEntityBean");
> >
> > Does that seem reasonable & non-confusing? Am a tad conscious of
> > adding too much magic under the covers - but in many ways conceptually
> > I kinda think when someone uses the code
> >
> >   exception(that).to(somewhere);
> >
> > they are kinda saying 'on that exception, go to somewhere" - which
> > kinda implies there'd be no retry logic applied by default - unless
> > you explicitly say you want that.
> >
> > --
> > James
> > -------
> > http://macstrac.blogspot.com/
> >
> >
>
> We also need the ability to do a 'retry until'.  Something like:
>
>
> exception(DBConnectionFailed.class).retryUntil(predicate);
>
>
> For some types of system events we just wish to retry forever but with the
> ability to kill the retry with an event if necessary for instance. The
> predicate could be connected to an mbean to allow remote control of hung
> retries etc...

Great idea - I've raised a JIRA for this...
https://issues.apache.org/activemq/browse/CAMEL-102


-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Nick Outram <ap...@nickoutram.name>]

James.Strachan wrote:
> 
> Here's a thought. The default error handler used in Camel is the
> DeadLetterChannel; in an attempt to do what most people want out of
> the box without having to explicitly customize things.
> 
> So most errors tend to be transient system issues (e.g. database
> deadlocks, temporary unavailability of connections/dependent
> services), so retrying N times before sending to a dead letter channel
> is a reasonable default.
> 
> When a user customizes certain exceptions in a route; its often gonna
> be a business exception or something similar. e.g. validation
> exceptions (whether XSD or business logic constraint violations etc).
> Usually for these cases, you don't want redelivery at all - since the
> exception isn't gonna change as its business logic.
> 
> So I was thinking, the default case should be that specifying an
> exception handler route - without doing anything else - would
> effectively by default, turn off retry for that exception, unless the
> user explicitly customizes the retry count for that exception - or
> maybe the user customizes the default value for all exception()
> policies.
> 
> e.g.
> 
> exception(Validation.class).to("seda:badlyValidatedMessages");
> from("activemq:foo").to("jpa:MyEntityBean");
> 
> if we get a database exception, we retry N times. However by default
> if we get a validation error we don't retry and just immediately send
> to "seda:badlyValidatedMessages".
> 
> If you don't want this, you can still be explicit...
> 
> // lets retry for this exception...
> exception(NullPointerException.class).maximumRetries(3);
> 
> // otherwise lets not retry for validations...
> exception(Validation.class).to("seda:badlyValidatedMessages");
> 
> from("activemq:foo").to("jpa:MyEntityBean");
> 
> Does that seem reasonable & non-confusing? Am a tad conscious of
> adding too much magic under the covers - but in many ways conceptually
> I kinda think when someone uses the code
> 
>   exception(that).to(somewhere);
> 
> they are kinda saying 'on that exception, go to somewhere" - which
> kinda implies there'd be no retry logic applied by default - unless
> you explicitly say you want that.
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> 

We also need the ability to do a 'retry until'.  Something like:


exception(DBConnectionFailed.class).retryUntil(predicate); 


For some types of system events we just wish to retry forever but with the
ability to kill the retry with an event if necessary for instance. The
predicate could be connected to an mbean to allow remote control of hung
retries etc...

-N

-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12141976
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/10/07, James Strachan <ja...@gmail.com> wrote:
> Here's a thought. The default error handler used in Camel is the
> DeadLetterChannel; in an attempt to do what most people want out of
> the box without having to explicitly customize things.
>
> So most errors tend to be transient system issues (e.g. database
> deadlocks, temporary unavailability of connections/dependent
> services), so retrying N times before sending to a dead letter channel
> is a reasonable default.
>
> When a user customizes certain exceptions in a route; its often gonna
> be a business exception or something similar. e.g. validation
> exceptions (whether XSD or business logic constraint violations etc).
> Usually for these cases, you don't want redelivery at all - since the
> exception isn't gonna change as its business logic.
>
> So I was thinking, the default case should be that specifying an
> exception handler route - without doing anything else - would
> effectively by default, turn off retry for that exception, unless the
> user explicitly customizes the retry count for that exception - or
> maybe the user customizes the default value for all exception()
> policies.
>
> e.g.
>
> exception(Validation.class).to("seda:badlyValidatedMessages");
> from("activemq:foo").to("jpa:MyEntityBean");
>
> if we get a database exception, we retry N times. However by default
> if we get a validation error we don't retry and just immediately send
> to "seda:badlyValidatedMessages".
>
> If you don't want this, you can still be explicit...
>
> // lets retry for this exception...
> exception(NullPointerException.class).maximumRetries(3);
>
> // otherwise lets not retry for validations...
> exception(Validation.class).to("seda:badlyValidatedMessages");
>
> from("activemq:foo").to("jpa:MyEntityBean");
>
> Does that seem reasonable & non-confusing? Am a tad conscious of
> adding too much magic under the covers - but in many ways conceptually
> I kinda think when someone uses the code
>
>   exception(that).to(somewhere);
>
> they are kinda saying 'on that exception, go to somewhere" - which
> kinda implies there'd be no retry logic applied by default - unless
> you explicitly say you want that.

FWIW this logic has made it into the forthcoming 1.1 release (CAMEL-97).

If you don't specify a redeliveryPolicy at all for a specific
exception(class) clause then no retries is assumed; otherwise your
custom redeliveryPolicy overloads the default redelivery policy.

So you can just overload, say, the maximumRedeliveries() or a specific
exception type - while inheriting all the other values etc. For
example see this test case...

https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/RedeliveryPolicyPerExceptionTest.java

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

Here's a thought. The default error handler used in Camel is the
DeadLetterChannel; in an attempt to do what most people want out of
the box without having to explicitly customize things.

So most errors tend to be transient system issues (e.g. database
deadlocks, temporary unavailability of connections/dependent
services), so retrying N times before sending to a dead letter channel
is a reasonable default.

When a user customizes certain exceptions in a route; its often gonna
be a business exception or something similar. e.g. validation
exceptions (whether XSD or business logic constraint violations etc).
Usually for these cases, you don't want redelivery at all - since the
exception isn't gonna change as its business logic.

So I was thinking, the default case should be that specifying an
exception handler route - without doing anything else - would
effectively by default, turn off retry for that exception, unless the
user explicitly customizes the retry count for that exception - or
maybe the user customizes the default value for all exception()
policies.

e.g.

exception(Validation.class).to("seda:badlyValidatedMessages");
from("activemq:foo").to("jpa:MyEntityBean");

if we get a database exception, we retry N times. However by default
if we get a validation error we don't retry and just immediately send
to "seda:badlyValidatedMessages".

If you don't want this, you can still be explicit...

// lets retry for this exception...
exception(NullPointerException.class).maximumRetries(3);

// otherwise lets not retry for validations...
exception(Validation.class).to("seda:badlyValidatedMessages");

from("activemq:foo").to("jpa:MyEntityBean");

Does that seem reasonable & non-confusing? Am a tad conscious of
adding too much magic under the covers - but in many ways conceptually
I kinda think when someone uses the code

  exception(that).to(somewhere);

they are kinda saying 'on that exception, go to somewhere" - which
kinda implies there'd be no retry logic applied by default - unless
you explicitly say you want that.

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/9/07, Neil Thorne <mr...@hotmail.com> wrote:
>
> Awesome stuff James.

Thanks for all the great feedback! :)


> The idea of aspect oriented routes is very nice.

Agreed!


> In our current use cases we can classify the results of a endpoint delivery
> into a few types.
>
> Success, exception1, exception2.
>
> The actions taking place on these exceptions are common.

I can imagine others having that kinda scenario too. e.g. doing
standard things with XML parsing errors/validation errors on messages
etc.



> We're found that we can classify our endpoint components according to a QoS
> type of
>
> For idempotent endpoints
>
> intercept(error(SystemException.class)).maximumRedeliveries(-1)
>
> for "never retry - we need to manually intervene first"
>
> intercept(error(BusinessException.class)).to("jms:errors")
>
> how can we register these interceptors across multiple routes?

So the nice thing is, stuff thats registered outside of a specific
route, is shared by all of the routes defined by that RouteBuilder.

So if you do all your interceptors/error handlers up front, before you
define any routes - the error handlers / interceptors are shared on
all routes.

If you want to customize this; you can overload the error handler
later on for other routes. Or its probably easier to just write
another RouteBuilder maybe?


> intercept(error(BusinessException.class)).to("jms:business.errors");
> intercept(error(SystemException.class)).maximumRedeliveries(-1);
> intercept(error(*)).to("jms:uncaught.errors");//not sure how to specify a
> catch all here
> from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");
>
> Read as:
>
> along the route bus->dateservice->paymentservice->responseservice
>
> if at any time a BusinessException occurs terminate the originating route
> flow and redirect to business.errors.
> if at any time a SystemException occurs retry until you can proceed
> if at any time any other exception occurs redirect to uncaught.errors
>
> In our use case the payment service is a "retry when its a system exception
> type", and a "never retry if it's a BusinessException type".
> Both of the other services are idempotent and will always throw
> SystemExceptions or potentially uncaught exceptions.
>
> Your new AOP concept should handle this nicely. We've got a clean route, and
> the error handling aspect is nicely separated.
>
> Endpoints can be categorized in a few different ways in our app.
>
> * by the exeptions they throw
> * by the retry policies (which can differ based on some exception type)
> * by the kind of data they expect in and out (Message Translation)
>
> We also do some common message translation too.
>
> You've already talked about some kind of registry. What about a registry of
> MessageTranslators registered by to and from types.

BTW have you seen the registry of type converters?

http://activemq.apache.org/camel/type-converter.html

which is great at converting the body or a header into a well defined
type you expect. It works great for different Classes (e.g. if you had
a POJO representation of a business object, then wanted to turn it
into a String or DOM or whatever). It doesn't work too well on more
general-formatting/translation issues though. e.g. what if you have a
number of different ways of marshalling a POJO to a stream; the
existing type converter mechanism isn't quite ideal for that.

So maybe we need a way to just define a registry of transformers
maybe? Which could have a route to define exactly how they work (so
they are kinda their own route?).


> Eg. I could have a route defined as
>
> from("jms:bus").translate(from("canonical").into("component").andBack()).
> to("jms:dateservice");
> from("jms:dateservice").translate(from("canonical").into("component").andBack()).
> to("jms:paymentservice");
> from("jms:paymentservice").translate(from("canonical").into("component").andBack()).
> to("jms:responseservice")
>
> If all of these are the same then I can use AOP style:
> //leave out the jms:bus endpoint
> intercept(except(endpoints("jms:bus")).translate(from("canonical").into("component").andBack());
> from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");
>
> that means all of my endpoints have message translation to and from the
> canonical format into component format along the route.

Interesting idea. BTW what does ".andBack()" mean? Also "canonical"
and "component" are just named formats right? So you could configure
some route/translation that goes from A, B or C to X etc.


> I can then easily change one translation for a new component:
>
> intercept(endpoints("jms:dateservice",
> "jms:responseservice")).translate(from("canonical").into("component").andBack());
> intercept(endpoints("jms:paymentservice").translate(from("canonical").into("componentNew").andBack());
> from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");
>
> We'd then just need to register the translators somewhere
>
> translate(from("component").to("canonical").andBack().with(componentToCanonicalMessageTranslator)));
> translate(from("componentNew").to("canonical").andBack().with(componentNewToCanonicalMessageTranslator)));

Interesting; its been one of those things on my list of things to
ponder about, how to register custom translators easily etc.


> intercept(endpoints("jms:dateservice",
> "jms:responseservice")).translate(from("canonical").into("component").andBack());
> intercept(endpoints("jms:paymentservice").translate(from("canonical").into("componentNew").andBack());
>
> from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");
>
> the translator could be given access to the appropriate part of the exchange
> body by the framework.
>
> hopefully some of this makes sense.

Yes! Nice work. Still mulling over the translator stuff; I'll post
more when I've had time to digest more fully

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Neil Thorne <mr...@hotmail.com>]

Awesome stuff James.

The idea of aspect oriented routes is very nice.

In our current use cases we can classify the results of a endpoint delivery
into a few types.

Success, exception1, exception2. 

The actions taking place on these exceptions are common.

We're found that we can classify our endpoint components according to a QoS
type of 

For idempotent endpoints 

intercept(error(SystemException.class)).maximumRedeliveries(-1)

for "never retry - we need to manually intervene first"

intercept(error(BusinessException.class)).to("jms:errors")

how can we register these interceptors across multiple routes?

intercept(error(BusinessException.class)).to("jms:business.errors");
intercept(error(SystemException.class)).maximumRedeliveries(-1);
intercept(error(*)).to("jms:uncaught.errors");//not sure how to specify a
catch all here
from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");

Read as:

along the route bus->dateservice->paymentservice->responseservice

if at any time a BusinessException occurs terminate the originating route
flow and redirect to business.errors.
if at any time a SystemException occurs retry until you can proceed
if at any time any other exception occurs redirect to uncaught.errors  

In our use case the payment service is a "retry when its a system exception
type", and a "never retry if it's a BusinessException type".
Both of the other services are idempotent and will always throw
SystemExceptions or potentially uncaught exceptions.

Your new AOP concept should handle this nicely. We've got a clean route, and
the error handling aspect is nicely separated.

Endpoints can be categorized in a few different ways in our app.

* by the exeptions they throw
* by the retry policies (which can differ based on some exception type)
* by the kind of data they expect in and out (Message Translation)

We also do some common message translation too.

You've already talked about some kind of registry. What about a registry of
MessageTranslators registered by to and from types.

Eg. I could have a route defined as

from("jms:bus").translate(from("canonical").into("component").andBack()).
to("jms:dateservice");
from("jms:dateservice").translate(from("canonical").into("component").andBack()).
to("jms:paymentservice");
from("jms:paymentservice").translate(from("canonical").into("component").andBack()).
to("jms:responseservice")

If all of these are the same then I can use AOP style:
//leave out the jms:bus endpoint
intercept(except(endpoints("jms:bus")).translate(from("canonical").into("component").andBack());
from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");

that means all of my endpoints have message translation to and from the
canonical format into component format along the route.

I can then easily change one translation for a new component:

intercept(endpoints("jms:dateservice",
"jms:responseservice")).translate(from("canonical").into("component").andBack());
intercept(endpoints("jms:paymentservice").translate(from("canonical").into("componentNew").andBack());
from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");

We'd then just need to register the translators somewhere

translate(from("component").to("canonical").andBack().with(componentToCanonicalMessageTranslator)));
translate(from("componentNew").to("canonical").andBack().with(componentNewToCanonicalMessageTranslator)));

intercept(endpoints("jms:dateservice",
"jms:responseservice")).translate(from("canonical").into("component").andBack());
intercept(endpoints("jms:paymentservice").translate(from("canonical").into("componentNew").andBack());

from("jms:bus").to("jms:dateservice").to("jms:paymentservice").to("jms.responseservice");

the translator could be given access to the appropriate part of the exchange
body by the framework.

hopefully some of this makes sense.

thanks,

Neil


James.Strachan wrote:
> 
> On 8/9/07, James Strachan <ja...@gmail.com> wrote:
>> On 8/9/07, James Strachan <ja...@gmail.com> wrote:
>> > On 8/9/07, Nick Outram <ap...@nickoutram.name> wrote:
>> > >
>> > >
>> > > James.Strachan wrote:
>> > > >
>> > > >
>> > > > Am just wondering the semantics of onPattern() really; as when I
>> first
>> > > > read your suggestion, in my mind I was thinking that onPattern()
>> would
>> > > > setup an interceptor. So maybe onPattern() needs some kind of
>> > > > proceed() call in part of its route to indicate if/when it carries
>> on
>> > > > processing?
>> > > >
>> > > > e.g. how about this (making a slight change to the language)...
>> > > >
>> > > >
>> onIntercept(anyPredicateGoesHere).to("seda:a").filter(somethingElse).proceed();
>> > > >
>> > > > from("seda:foo").to("seda:b");
>> > > >
>> > > > i.e. in the above we intercept the route, if the message matches
>> > > > anyPredicateGoesHere, then its sent to "seda:a" and if
>> somethingElse
>> > > > matches, then it carries on to "seda:b". So it could go to
>> "seda:a",
>> > > > "seda:b" or both - depending on the evaluation of the two
>> predicates.
>> > > >
>> > > > If you really just want it to go to "seda:a" or "seda:b" then you'd
>> just
>> > > > do...
>> > > >
>> > > > onIntercept(anyPredicateGoesHere).to("seda:a");
>> > > >
>> > > > from("seda:foo").to("seda:b");
>> > > >
>> > > > In both cases really, we're just putting Message Filter / Content
>> > > > Based Router inside an interceptor; rather than in-place inside the
>> > > > route (and allowing the default behaviour of interceptors to be
>> > > > inherited on all child nodes of the DSL, so the interceptor would
>> be
>> > > > applied around all Processors in the chain).
>> > > >
>> > > > How's that sound?
>> > > >
>> > > >
>> > > >
>> > >
>> > >
>> > > onIntercept().choice()
>> > >                 .when(anyPredicateGoesHere).to("seda:a")
>> > >                 .when(somethingElse).to("seda:b", "seda:c")
>> > >                 .when(somethingElseAgain).to("seda:d", proceed());
>> > >
>> > > from("seda:foo").process(new Enricher1()). process(new
>> > > Enricher2()).to("seda:z");
>> > >
>> > > Reads:
>> > > After from("seda:foo") or process(new Enricher1()) or process(new
>> > > Enricher2()) if the message matches anyPredicateGoesHere then send to
>> > > seda:a, if the message matches somethinElse then send it to seda:b
>> and
>> > > seda:c, if the message matches somethingElseAgain send it to seda:d
>> and
>> > > proceed on the current route, otherwise in all other cases just
>> proceed; you
>> > > could include an explicit otherwise clause but as this catches
>> everything it
>> > > would make your route pretty redundant.
>> > >
>> > > Also:
>> > >
>> > >
>> onIntercept().filter(xpath("/person[@name='James']")).to("mock:result",
>> > > proceed());
>> > >
>> > > … after any processing on the route if the xpath matches then send
>> the
>> > > message to mock:result and proceed on the original route…
>> > >
>> > > and even:
>> > >
>> > >
>> onIntercept().aggregator(header("JMSDestination")).to("activemq:someSlowTopicForGuis");
>> > >
>> > >
>> > > Basically the onIntercept() applies the following message routing
>> rules to
>> > > all processes on routes it is registered with.
>> > >
>> > > If you really wanted to have fun you could have:
>> > >
>> > >
>> onIntercept(interceptCriteria).filter(xpath("/person[@name='James']")).to("mock:result",
>> > > proceed());
>> > >
>> > > Which would only apply the intercept to processors matching the
>> > > interceptCriteria…
>> > >
>> > > Thoughts? Am I raving ;-)
>> >
>> > Love it! :) Thanks for this awesome feedback!
>> >
>> > I'll see if I can have a go at implementing this later today unless
>> > anyone beats me to it...
>>
>> OK I've a first cut of the interceptor route definition. BTW I raised
>> a JIRA to track this...
>> https://issues.apache.org/activemq/browse/CAMEL-92
>>
>> When actually implementing it, I found the method 'intercept()' to fit
>> better with the existing code; so went with that to start with - is
>> that OK with you?
>>
>> Now in a route builder if you do
>>
>>   intercept().
>>
>> you are now defining an interceptor route, you must call proceed() to
>> continue processing the underlying route. e.g.
>>
>>   intercept().to("foo")
>>
>> would kinda replace the routing rules you have - so won't be too useful
>> :)
>>
>> also if you do
>>
>>   intercept(Predicate)
>>
>> the same rules apply - though if the predicate is false then there's a
>> kinda explicit call to proceed() for you. If the predicate is true,
>> its up to you to decide when/if to proceed() etc.
>>
>> Probably the easiest way to check I've implemented what you were
>> thinking is to try the test cases; they are all here beginning with
>> "Intercept*.java"
>> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/
>>
>> i.e.
>> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptRouteTest.java
>> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithPredicateAndProceedRouteTest.java
>> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithPredicateRouteTest.java
>> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithoutProceedRouteTest.java
>>
>>
>> Does this seem OK to you? Will start on the onError() now...
> 
> OK the onError() is done now too. Again when I actually came to code
> it, I found it was a tad better to call it exception().
> 
> So you can now do
> 
> exception(FooException.class).to("seda:whatnot");
> 
> from("foo").process(something).to("seda:bar");
> 
> if the FooException is thrown then its sent to the "seda:whatnot"
> queue; otherwise the default error handling takes place.
> 
> You can see this DSL in action using a variation of the ValidationTest...
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ValidationWithExceptionTest.java
> 
> I'll ponder how easy its gonna be to use the DSL to customize
> redelivery policy stuff on a per type basis too...
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> 

-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12082262
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/9/07, James Strachan <ja...@gmail.com> wrote:
> On 8/9/07, James Strachan <ja...@gmail.com> wrote:
> > On 8/9/07, Nick Outram <ap...@nickoutram.name> wrote:
> > >
> > >
> > > James.Strachan wrote:
> > > >
> > > >
> > > > Am just wondering the semantics of onPattern() really; as when I first
> > > > read your suggestion, in my mind I was thinking that onPattern() would
> > > > setup an interceptor. So maybe onPattern() needs some kind of
> > > > proceed() call in part of its route to indicate if/when it carries on
> > > > processing?
> > > >
> > > > e.g. how about this (making a slight change to the language)...
> > > >
> > > > onIntercept(anyPredicateGoesHere).to("seda:a").filter(somethingElse).proceed();
> > > >
> > > > from("seda:foo").to("seda:b");
> > > >
> > > > i.e. in the above we intercept the route, if the message matches
> > > > anyPredicateGoesHere, then its sent to "seda:a" and if somethingElse
> > > > matches, then it carries on to "seda:b". So it could go to "seda:a",
> > > > "seda:b" or both - depending on the evaluation of the two predicates.
> > > >
> > > > If you really just want it to go to "seda:a" or "seda:b" then you'd just
> > > > do...
> > > >
> > > > onIntercept(anyPredicateGoesHere).to("seda:a");
> > > >
> > > > from("seda:foo").to("seda:b");
> > > >
> > > > In both cases really, we're just putting Message Filter / Content
> > > > Based Router inside an interceptor; rather than in-place inside the
> > > > route (and allowing the default behaviour of interceptors to be
> > > > inherited on all child nodes of the DSL, so the interceptor would be
> > > > applied around all Processors in the chain).
> > > >
> > > > How's that sound?
> > > >
> > > >
> > > >
> > >
> > >
> > > onIntercept().choice()
> > >                 .when(anyPredicateGoesHere).to("seda:a")
> > >                 .when(somethingElse).to("seda:b", "seda:c")
> > >                 .when(somethingElseAgain).to("seda:d", proceed());
> > >
> > > from("seda:foo").process(new Enricher1()). process(new
> > > Enricher2()).to("seda:z");
> > >
> > > Reads:
> > > After from("seda:foo") or process(new Enricher1()) or process(new
> > > Enricher2()) if the message matches anyPredicateGoesHere then send to
> > > seda:a, if the message matches somethinElse then send it to seda:b and
> > > seda:c, if the message matches somethingElseAgain send it to seda:d and
> > > proceed on the current route, otherwise in all other cases just proceed; you
> > > could include an explicit otherwise clause but as this catches everything it
> > > would make your route pretty redundant.
> > >
> > > Also:
> > >
> > > onIntercept().filter(xpath("/person[@name='James']")).to("mock:result",
> > > proceed());
> > >
> > > … after any processing on the route if the xpath matches then send the
> > > message to mock:result and proceed on the original route…
> > >
> > > and even:
> > >
> > > onIntercept().aggregator(header("JMSDestination")).to("activemq:someSlowTopicForGuis");
> > >
> > >
> > > Basically the onIntercept() applies the following message routing rules to
> > > all processes on routes it is registered with.
> > >
> > > If you really wanted to have fun you could have:
> > >
> > > onIntercept(interceptCriteria).filter(xpath("/person[@name='James']")).to("mock:result",
> > > proceed());
> > >
> > > Which would only apply the intercept to processors matching the
> > > interceptCriteria…
> > >
> > > Thoughts? Am I raving ;-)
> >
> > Love it! :) Thanks for this awesome feedback!
> >
> > I'll see if I can have a go at implementing this later today unless
> > anyone beats me to it...
>
> OK I've a first cut of the interceptor route definition. BTW I raised
> a JIRA to track this...
> https://issues.apache.org/activemq/browse/CAMEL-92
>
> When actually implementing it, I found the method 'intercept()' to fit
> better with the existing code; so went with that to start with - is
> that OK with you?
>
> Now in a route builder if you do
>
>   intercept().
>
> you are now defining an interceptor route, you must call proceed() to
> continue processing the underlying route. e.g.
>
>   intercept().to("foo")
>
> would kinda replace the routing rules you have - so won't be too useful :)
>
> also if you do
>
>   intercept(Predicate)
>
> the same rules apply - though if the predicate is false then there's a
> kinda explicit call to proceed() for you. If the predicate is true,
> its up to you to decide when/if to proceed() etc.
>
> Probably the easiest way to check I've implemented what you were
> thinking is to try the test cases; they are all here beginning with
> "Intercept*.java"
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/
>
> i.e.
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptRouteTest.java
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithPredicateAndProceedRouteTest.java
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithPredicateRouteTest.java
> https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithoutProceedRouteTest.java
>
>
> Does this seem OK to you? Will start on the onError() now...

OK the onError() is done now too. Again when I actually came to code
it, I found it was a tad better to call it exception().

So you can now do

exception(FooException.class).to("seda:whatnot");

from("foo").process(something).to("seda:bar");

if the FooException is thrown then its sent to the "seda:whatnot"
queue; otherwise the default error handling takes place.

You can see this DSL in action using a variation of the ValidationTest...
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/ValidationWithExceptionTest.java

I'll ponder how easy its gonna be to use the DSL to customize
redelivery policy stuff on a per type basis too...

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/9/07, James Strachan <ja...@gmail.com> wrote:
> On 8/9/07, Nick Outram <ap...@nickoutram.name> wrote:
> >
> >
> > James.Strachan wrote:
> > >
> > >
> > > Am just wondering the semantics of onPattern() really; as when I first
> > > read your suggestion, in my mind I was thinking that onPattern() would
> > > setup an interceptor. So maybe onPattern() needs some kind of
> > > proceed() call in part of its route to indicate if/when it carries on
> > > processing?
> > >
> > > e.g. how about this (making a slight change to the language)...
> > >
> > > onIntercept(anyPredicateGoesHere).to("seda:a").filter(somethingElse).proceed();
> > >
> > > from("seda:foo").to("seda:b");
> > >
> > > i.e. in the above we intercept the route, if the message matches
> > > anyPredicateGoesHere, then its sent to "seda:a" and if somethingElse
> > > matches, then it carries on to "seda:b". So it could go to "seda:a",
> > > "seda:b" or both - depending on the evaluation of the two predicates.
> > >
> > > If you really just want it to go to "seda:a" or "seda:b" then you'd just
> > > do...
> > >
> > > onIntercept(anyPredicateGoesHere).to("seda:a");
> > >
> > > from("seda:foo").to("seda:b");
> > >
> > > In both cases really, we're just putting Message Filter / Content
> > > Based Router inside an interceptor; rather than in-place inside the
> > > route (and allowing the default behaviour of interceptors to be
> > > inherited on all child nodes of the DSL, so the interceptor would be
> > > applied around all Processors in the chain).
> > >
> > > How's that sound?
> > >
> > >
> > >
> >
> >
> > onIntercept().choice()
> >                 .when(anyPredicateGoesHere).to("seda:a")
> >                 .when(somethingElse).to("seda:b", "seda:c")
> >                 .when(somethingElseAgain).to("seda:d", proceed());
> >
> > from("seda:foo").process(new Enricher1()). process(new
> > Enricher2()).to("seda:z");
> >
> > Reads:
> > After from("seda:foo") or process(new Enricher1()) or process(new
> > Enricher2()) if the message matches anyPredicateGoesHere then send to
> > seda:a, if the message matches somethinElse then send it to seda:b and
> > seda:c, if the message matches somethingElseAgain send it to seda:d and
> > proceed on the current route, otherwise in all other cases just proceed; you
> > could include an explicit otherwise clause but as this catches everything it
> > would make your route pretty redundant.
> >
> > Also:
> >
> > onIntercept().filter(xpath("/person[@name='James']")).to("mock:result",
> > proceed());
> >
> > … after any processing on the route if the xpath matches then send the
> > message to mock:result and proceed on the original route…
> >
> > and even:
> >
> > onIntercept().aggregator(header("JMSDestination")).to("activemq:someSlowTopicForGuis");
> >
> >
> > Basically the onIntercept() applies the following message routing rules to
> > all processes on routes it is registered with.
> >
> > If you really wanted to have fun you could have:
> >
> > onIntercept(interceptCriteria).filter(xpath("/person[@name='James']")).to("mock:result",
> > proceed());
> >
> > Which would only apply the intercept to processors matching the
> > interceptCriteria…
> >
> > Thoughts? Am I raving ;-)
>
> Love it! :) Thanks for this awesome feedback!
>
> I'll see if I can have a go at implementing this later today unless
> anyone beats me to it...

OK I've a first cut of the interceptor route definition. BTW I raised
a JIRA to track this...
https://issues.apache.org/activemq/browse/CAMEL-92

When actually implementing it, I found the method 'intercept()' to fit
better with the existing code; so went with that to start with - is
that OK with you?

Now in a route builder if you do

  intercept().

you are now defining an interceptor route, you must call proceed() to
continue processing the underlying route. e.g.

  intercept().to("foo")

would kinda replace the routing rules you have - so won't be too useful :)

also if you do

  intercept(Predicate)

the same rules apply - though if the predicate is false then there's a
kinda explicit call to proceed() for you. If the predicate is true,
its up to you to decide when/if to proceed() etc.

Probably the easiest way to check I've implemented what you were
thinking is to try the test cases; they are all here beginning with
"Intercept*.java"
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/

i.e.
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptRouteTest.java
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithPredicateAndProceedRouteTest.java
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithPredicateRouteTest.java
https://svn.apache.org/repos/asf/activemq/camel/trunk/camel-core/src/test/java/org/apache/camel/processor/InterceptWithoutProceedRouteTest.java


Does this seem OK to you? Will start on the onError() now...

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/9/07, Nick Outram <ap...@nickoutram.name> wrote:
>
>
>
> James.Strachan wrote:
> >
> >
> > I'll see if I can have a go at implementing this later today unless
> > anyone beats me to it...
> > --
> > James
> > -------
> > http://macstrac.blogspot.com/
> >
> >
>
> 1.0 or 1.1? Just a bit confused over the release status of Camel...

1.0 has been out a while - I'm hoping to start cutting the 1.1 release
very soon (e.g. today!)
-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Nick Outram <ap...@nickoutram.name>]

James.Strachan wrote:
> 
> 
> I'll see if I can have a go at implementing this later today unless
> anyone beats me to it...
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> 

1.0 or 1.1? Just a bit confused over the release status of Camel...

-N
-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12071785
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/9/07, Nick Outram <ap...@nickoutram.name> wrote:
>
>
> James.Strachan wrote:
> >
> >
> > Am just wondering the semantics of onPattern() really; as when I first
> > read your suggestion, in my mind I was thinking that onPattern() would
> > setup an interceptor. So maybe onPattern() needs some kind of
> > proceed() call in part of its route to indicate if/when it carries on
> > processing?
> >
> > e.g. how about this (making a slight change to the language)...
> >
> > onIntercept(anyPredicateGoesHere).to("seda:a").filter(somethingElse).proceed();
> >
> > from("seda:foo").to("seda:b");
> >
> > i.e. in the above we intercept the route, if the message matches
> > anyPredicateGoesHere, then its sent to "seda:a" and if somethingElse
> > matches, then it carries on to "seda:b". So it could go to "seda:a",
> > "seda:b" or both - depending on the evaluation of the two predicates.
> >
> > If you really just want it to go to "seda:a" or "seda:b" then you'd just
> > do...
> >
> > onIntercept(anyPredicateGoesHere).to("seda:a");
> >
> > from("seda:foo").to("seda:b");
> >
> > In both cases really, we're just putting Message Filter / Content
> > Based Router inside an interceptor; rather than in-place inside the
> > route (and allowing the default behaviour of interceptors to be
> > inherited on all child nodes of the DSL, so the interceptor would be
> > applied around all Processors in the chain).
> >
> > How's that sound?
> >
> >
> >
>
>
> onIntercept().choice()
>                 .when(anyPredicateGoesHere).to("seda:a")
>                 .when(somethingElse).to("seda:b", "seda:c")
>                 .when(somethingElseAgain).to("seda:d", proceed());
>
> from("seda:foo").process(new Enricher1()). process(new
> Enricher2()).to("seda:z");
>
> Reads:
> After from("seda:foo") or process(new Enricher1()) or process(new
> Enricher2()) if the message matches anyPredicateGoesHere then send to
> seda:a, if the message matches somethinElse then send it to seda:b and
> seda:c, if the message matches somethingElseAgain send it to seda:d and
> proceed on the current route, otherwise in all other cases just proceed; you
> could include an explicit otherwise clause but as this catches everything it
> would make your route pretty redundant.
>
> Also:
>
> onIntercept().filter(xpath("/person[@name='James']")).to("mock:result",
> proceed());
>
> … after any processing on the route if the xpath matches then send the
> message to mock:result and proceed on the original route…
>
> and even:
>
> onIntercept().aggregator(header("JMSDestination")).to("activemq:someSlowTopicForGuis");
>
>
> Basically the onIntercept() applies the following message routing rules to
> all processes on routes it is registered with.
>
> If you really wanted to have fun you could have:
>
> onIntercept(interceptCriteria).filter(xpath("/person[@name='James']")).to("mock:result",
> proceed());
>
> Which would only apply the intercept to processors matching the
> interceptCriteria…
>
> Thoughts? Am I raving ;-)

Love it! :) Thanks for this awesome feedback!

I'll see if I can have a go at implementing this later today unless
anyone beats me to it...
-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Nick Outram <ap...@nickoutram.name>]

James.Strachan wrote:
> 
> 
> Am just wondering the semantics of onPattern() really; as when I first
> read your suggestion, in my mind I was thinking that onPattern() would
> setup an interceptor. So maybe onPattern() needs some kind of
> proceed() call in part of its route to indicate if/when it carries on
> processing?
> 
> e.g. how about this (making a slight change to the language)...
> 
> onIntercept(anyPredicateGoesHere).to("seda:a").filter(somethingElse).proceed();
> 
> from("seda:foo").to("seda:b");
> 
> i.e. in the above we intercept the route, if the message matches
> anyPredicateGoesHere, then its sent to "seda:a" and if somethingElse
> matches, then it carries on to "seda:b". So it could go to "seda:a",
> "seda:b" or both - depending on the evaluation of the two predicates.
> 
> If you really just want it to go to "seda:a" or "seda:b" then you'd just
> do...
> 
> onIntercept(anyPredicateGoesHere).to("seda:a");
> 
> from("seda:foo").to("seda:b");
> 
> In both cases really, we're just putting Message Filter / Content
> Based Router inside an interceptor; rather than in-place inside the
> route (and allowing the default behaviour of interceptors to be
> inherited on all child nodes of the DSL, so the interceptor would be
> applied around all Processors in the chain).
> 
> How's that sound?
> 
> 
> 


onIntercept().choice() 
                .when(anyPredicateGoesHere).to("seda:a") 
                .when(somethingElse).to("seda:b", “seda:c”) 
                .when(somethingElseAgain).to("seda:d", proceed());

from("seda:foo").process(new Enricher1()). process(new
Enricher2()).to("seda:z”);

Reads:
After from("seda:foo") or process(new Enricher1()) or process(new
Enricher2()) if the message matches anyPredicateGoesHere then send to
seda:a, if the message matches somethinElse then send it to seda:b and
seda:c, if the message matches somethingElseAgain send it to seda:d and
proceed on the current route, otherwise in all other cases just proceed; you
could include an explicit otherwise clause but as this catches everything it
would make your route pretty redundant.

Also:

onIntercept().filter(xpath("/person[@name='James']")).to("mock:result",
proceed());

… after any processing on the route if the xpath matches then send the
message to mock:result and proceed on the original route…

and even:

onIntercept().aggregator(header("JMSDestination")).to("activemq:someSlowTopicForGuis");


Basically the onIntercept() applies the following message routing rules to
all processes on routes it is registered with.

If you really wanted to have fun you could have:

onIntercept(interceptCriteria).filter(xpath("/person[@name='James']")).to("mock:result",
proceed());

Which would only apply the intercept to processors matching the
interceptCriteria…

Thoughts? Am I raving ;-)

-N

-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12071407
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/9/07, Nick Outram <ap...@nickoutram.name> wrote:
> James.Strachan wrote:
> > I'm wondering about something like...
> >
> > // lets setup the error handler first
> > onError(MyException.class).maximumRedeliveries(5).onFail().to("seda:c")
> >
> > // now lets do the route
> > from("foo").to("bar");
> >
> > I guess if we define the error handler in the middle of a route we
> > might need some kinda escape expression to go back to the 'real route'
> > from the error handler route. e.g.
> >
> > // this is bad...
> > from("foo").
> >       onError(MyException.class).maximumRedeliveries(5).onFail().to("seda:c").
> >   to("bar"); // by default this would still be part of the error handler
> >
> > maybe something like
> >
> > from("foo").
> >       onError(MyException.class).
> >         maximumRedeliveries(5).onFail().to("seda:c").
> >       onError(Bar.class).
> >         to("seda:d")
> >       onSucccess().
> >         to("bar");
> >
> > i.e. the onError(Class) stuff starts adding an error route for a
> > specific type of exception (which also allows you to configure retry
> > policy & the dead letter destination etc)
> >
> > Thoughts?
> >
> >
>
> That works too. I particularly like the onError(... construct as it keeps
> the configuration of the error handling maxRedeliveries etc. out of the
> route definition 'from("foo").to("bar");'.

Yeah


> Not so hot on your second example for the same reason... in-lining the
> onError(... mixes routing semantics with configuration and makes the DSL
> messy.

Agreed - I think defining error handling & interceptors outside of the
route helps readability enormously - and hopefully will encourage the
reuse of error handling/interceptor code across routes; I guess I just
wondered what if folks did want to describe error handling inside a
route & how that could affect the 'grammar' of the DSL.


> It also strikes me that the onError(… approach could be further expanded to
> handle more generalised matches i.e.
>
> // lets setup the error handler first
> onPattern(error(MyException.class)).maximumRedeliveries(5).onFail().to("seda:c")
>
> // lets trap any messages with settlement amounts more than £100000
> onPattern(xpath("/settlementAmnt >10000").to("seda:authorisationRequired");
>
> // now lets do the route
> from("foo").process(new myEnricher()).to("bar");
>
> The 'onPattern(…' would trap matching messages anywhere on the route at
> 'node' transitions; sort of AOP for routing.

Interesting idea! :)

I guess the onError() is about registering an error handling route
(which in the implementation would hook into the ErrorHandler
implementation to process certain kinds of error using custom routes,
for others the default behaviour kicks in etc).

The onPattern(...) feels more about adding a kind of interceptor
around each processor in the route (which we support now BTW, though
your suggestion is a much nicer DSL for adding them). The error
handling is clear in my mind; since its an exception, the current
route terminates (however deeply nested) and then the error route
kicks in. For the onPattern() AOP thingy; how do we know whether or
not we should continue processing the original route?

i.e. in your example, would it just go to "seda:authorisationRequired"
if the predicate matches or to both destinations?

Am just wondering the semantics of onPattern() really; as when I first
read your suggestion, in my mind I was thinking that onPattern() would
setup an interceptor. So maybe onPattern() needs some kind of
proceed() call in part of its route to indicate if/when it carries on
processing?

e.g. how about this (making a slight change to the language)...

onIntercept(anyPredicateGoesHere).to("seda:a").filter(somethingElse).proceed();

from("seda:foo").to("seda:b");

i.e. in the above we intercept the route, if the message matches
anyPredicateGoesHere, then its sent to "seda:a" and if somethingElse
matches, then it carries on to "seda:b". So it could go to "seda:a",
"seda:b" or both - depending on the evaluation of the two predicates.

If you really just want it to go to "seda:a" or "seda:b" then you'd just do...

onIntercept(anyPredicateGoesHere).to("seda:a");

from("seda:foo").to("seda:b");

In both cases really, we're just putting Message Filter / Content
Based Router inside an interceptor; rather than in-place inside the
route (and allowing the default behaviour of interceptors to be
inherited on all child nodes of the DSL, so the interceptor would be
applied around all Processors in the chain).

How's that sound?


-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Nick Outram <ap...@nickoutram.name>]

James.Strachan wrote:
> 
> 
> I'm wondering about something like...
> 
> // lets setup the error handler first
> onError(MyException.class).maximumRedeliveries(5).onFail().to("seda:c")
> 
> // now lets do the route
> from("foo").to("bar");
> 
> I guess if we define the error handler in the middle of a route we
> might need some kinda escape expression to go back to the 'real route'
> from the error handler route. e.g.
> 
> // this is bad...
> from("foo").
> 	onError(MyException.class).maximumRedeliveries(5).onFail().to("seda:c").
>   to("bar"); // by default this would still be part of the error handler
> 
> maybe something like
> 
> from("foo").
> 	onError(MyException.class).
> 	  maximumRedeliveries(5).onFail().to("seda:c").
> 	onError(Bar.class).
> 	  to("seda:d")
> 	onSucccess().
> 	  to("bar");
> 
> i.e. the onError(Class) stuff starts adding an error route for a
> specific type of exception (which also allows you to configure retry
> policy & the dead letter destination etc)
> 
> Thoughts?
> 
> 

That works too. I particularly like the onError(... construct as it keeps
the configuration of the error handling maxRedeliveries etc. out of the
route definition 'from("foo").to("bar");'.

Not so hot on your second example for the same reason... in-lining the
onError(... mixes routing semantics with configuration and makes the DSL
messy.

It also strikes me that the onError(… approach could be further expanded to
handle more generalised matches i.e.

// lets setup the error handler first 
onPattern(error(MyException.class)).maximumRedeliveries(5).onFail().to("seda:c") 

// lets trap any messages with settlement amounts more than £100000
onPattern(xpath("/settlementAmnt >10000”).to(“seda:authorisationRequired”);

// now lets do the route 
from("foo").process(new myEnricher()).to("bar");

The ‘onPattern(…’ would trap matching messages anywhere on the route at
‘node’ transitions; sort of AOP for routing.

-N


-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12068759
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/7/07, Nick Outram <ap...@nickoutram.name> wrote:
> James.Strachan wrote:
> >
> > On 8/7/07, James Strachan <ja...@gmail.com> wrote:
> >
> > I've just added to trunk a little try/catch style routing DSL.
> >
> > In Java we can't use try/catch/finally words so I've initially gone with
> > this
> >
> >   from("direct:start").
> >           tryBlock().
> >               process(validator).
> >              to("mock:valid").
> >           handle(ValidationException.class).to("mock:invalid");
> >
> > which is maybe not the easiest thing to read; but tryBlock / handle is
> > a replacement for try/catch. (Maybe tryBlock/catchBlock is more
> > consistent?). Any name suggestions most welcome! (Its often the
> > hardest part of programming, coming up with good names for things).
> >
> > In XML its a little easier on the eye. e.g. here's a validation test
> > case to ensure that messages are delivered to different endpoints
> > depending on whether they are valid or not...
> > https://svn.apache.org/repos/asf/activemq/camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/component/validator/camelContext.xml
>
> Hmm, not sure about the try/catch approach :confused: as it makes the DSL a
> bit verbose. I think I'd prefer the implicit try/catch that we already have
> with the ErrorHandler; more of an AOP approach. How about using an
> ErrorRoute instead of an ErrorHandler? Once an error is on the ErrorRoute it
> has the full semantics of the DSL available for filtering/splitting etc.
>
> We'd have to add semantics to allow retry to an endpoint. Something like:
>
> from("error:start").
>           .choice()
>                 .when(error(ValidationException.class)).to("seda:a")
>                 .when(error(BusinessDataException.class)).to("seda:b")
>
> .when(error(SystemException.class)).retry().useCollisionAvoidance().maximumRedeliveries(0)
>                                 .useExponentialBackOff()
>
>
> .when(error(PooException.class)).retry().useCollisionAvoidance().maximumRedeliveries(-1)
>                                 .useExponentialBackOff()
>                 .otherwise().to("seda:c");
>
> Where the retry() will post the message back to the route that generated the
> error...
> I just threw the 'when(error(...' semantic in to make it explicit but maybe
> the exception class is already available in the context?
> As in this example I'd want to be able to separately configure the retry
> behaviour for different exceptions.

Interesting idea! Its certainly much cleaner than my attempt! :) I
think some folks might still want try/catch in some circumstances -
however I agree we need nicer error handling in the DSL.

I'm wondering about something like...

// lets setup the error handler first
onError(MyException.class).maximumRedeliveries(5).onFail().to("seda:c")

// now lets do the route
from("foo").to("bar");

I guess if we define the error handler in the middle of a route we
might need some kinda escape expression to go back to the 'real route'
from the error handler route. e.g.

// this is bad...
from("foo").
	onError(MyException.class).maximumRedeliveries(5).onFail().to("seda:c").
  to("bar"); // by default this would still be part of the error handler

maybe something like

from("foo").
	onError(MyException.class).
	  maximumRedeliveries(5).onFail().to("seda:c").
	onError(Bar.class).
	  to("seda:d")
	onSucccess().
	  to("bar");

i.e. the onError(Class) stuff starts adding an error route for a
specific type of exception (which also allows you to configure retry
policy & the dead letter destination etc)

Thoughts?

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Nick Outram <ap...@nickoutram.name>]

James.Strachan wrote:
> 
> On 8/7/07, James Strachan <ja...@gmail.com> wrote:
> 
> I've just added to trunk a little try/catch style routing DSL.
> 
> In Java we can't use try/catch/finally words so I've initially gone with
> this
> 
>   from("direct:start").
>           tryBlock().
>               process(validator).
>              to("mock:valid").
>           handle(ValidationException.class).to("mock:invalid");
> 
> which is maybe not the easiest thing to read; but tryBlock / handle is
> a replacement for try/catch. (Maybe tryBlock/catchBlock is more
> consistent?). Any name suggestions most welcome! (Its often the
> hardest part of programming, coming up with good names for things).
> 
> In XML its a little easier on the eye. e.g. here's a validation test
> case to ensure that messages are delivered to different endpoints
> depending on whether they are valid or not...
> https://svn.apache.org/repos/asf/activemq/camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/component/validator/camelContext.xml
> 
> In Java we could even use the real try/catch handling using an inline
> Processor
> 
> e.g.
> 
> 	fromC
> 
> 

Hmm, not sure about the try/catch approach :confused: as it makes the DSL a
bit verbose. I think I'd prefer the implicit try/catch that we already have
with the ErrorHandler; more of an AOP approach. How about using an
ErrorRoute instead of an ErrorHandler? Once an error is on the ErrorRoute it
has the full semantics of the DSL available for filtering/splitting etc.

We'd have to add semantics to allow retry to an endpoint. Something like:

from("error:start").
          .choice()
                .when(error(ValidationException.class)).to("seda:a")
                .when(error(BusinessDataException.class)).to("seda:b")
               
.when(error(SystemException.class)).retry().useCollisionAvoidance().maximumRedeliveries(0)
                                .useExponentialBackOff()

               
.when(error(PooException.class)).retry().useCollisionAvoidance().maximumRedeliveries(-1)
                                .useExponentialBackOff()
                .otherwise().to("seda:c");

Where the retry() will post the message back to the route that generated the
error...
I just threw the 'when(error(...' semantic in to make it explicit but maybe
the exception class is already available in the context?
As in this example I'd want to be able to separately configure the retry
behaviour for different exceptions.

Thoughts?
-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12038122
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/7/07, James Strachan <ja...@gmail.com> wrote:
> On 8/7/07, Neil Thorne <mr...@hotmail.com> wrote:
> >
> > Hi Neil
> > Hi James thanks for the quick reply.
> >
> > >> a) a system exception gets thrown. A system exception indicates an
> > >> infrastructure problem that will be fixed ASAP as it is essential to the
> > >> correct functioning of the system as a whole. eg. jdbc connection failed,
> > >> remote system is down etc. If a system exception is thrown then that
> > >> operation should be continually retried until it is successful.
> > >>
> > >> b) a business exception is thrown. A business exception indicates that
> > >> there
> > >> was a problem with the request itself. It should not be retried (at least
> > >> not until the problem has been resolved). The message should be moved
> > >> onto a
> > >> hospital queue where it can be inspected, changed and retried after
> > >> possible
> > >> manual intervention.
> >
> > > BTW how can you tell the difference between a) and b)? Are there
> > > certain base exception types you can use?
> >
> > Yes I can. Although our applications can throw quite a few different
> > exceptions, so far in terms of routing we only need these two base types. We
> > have translators that convert the original exception to one of these types.
>
> Ah cool. I was wondering; rather like the Type Converter stuff....
> http://activemq.apache.org/camel/type-converter.html
>
> Do we need a simple little framework where we can easily mark
> exception types as being 'system exceptions' or 'business exceptions';
> so the error handler can do the right thing by default without
> explicit configuration in the DSL?
>
> e.g. for code you own, we could add an @BusinessException or
> @SystemException annotation. The annotation could even configure the
> retry count/timeout to be specified on a per exception basis.
>
> For code that we don't own, we could add some kinda
> META-INF/services/org/apache/camel/BusinessExceptions file that lists
> the base classes of all the business level exceptions (or system
> exceptions etc).
>
> Am thinking btw that it might be easier to mark just the business
> level exceptions; as the system exceptions tend to be more diverse &
> harder to enumerate.
>
> If we could know using the 'ExceptionRepository' what exceptions are
> which, then we can know whether or not to retry etc.
>
> Am even thinking, maybe we could attach different error handling to
> certain types of exception. e.g. if we can ever really detect a true
> database deadlock (which is getting harder these days due to exception
> wrapping etc) - we might want to increase the retry counts etc.
>
>
> > >> What is the simplest way to achieve this with camel? Currently I can only
> > >> see an errorHandler which assumes you want to dump messages that cannot
> > >> be
> > >> processed to an error endpoint.
> >
> > >An ErrorHandler could do anything - e.g. based on the type of
> > >exception it could decide whether or not to retry (if at all) etc.
> >
> > >The default implementation is DeadLetterChannelErrorHandler, which
> > >assumes n-retries with optional backoff delays - before sending to a
> > >'dead letter' endpoint, which is kinda like your hospital queue
> >
> > Okay I can write my own implementation.
>
> Please let us know how you get on - as I'd love something like this to
> be in the core of Camel.
>
> e.g. maybe we can patch the default DeadLetterChannelErrorHandler to
> use some kinda pluggable strategy to determine if an exception is a
> business level exception (and so the retry logic is not used etc)
>
>
> > We're using the Java DSL style of
> > configuration.
> >
> > I'd need to set two different retry strategies depending on the type of
> > exception.
> >
> > from("direct:a").errorHandler().filter(
> > type.isEqualTo("com.acme.BusinessException").
> > maximumRedeliveries(0).to("direct:business.errors")
> > ).errorHandler().filter(
> > type.isEqualTo("com.acme.SystemException").
> > maximumRedeliveries(-1).useExponentialBackOff()
> > ).to("direct:b");
>
> Looks good :)
>
>
> > Without explicit DSL support I'm wondering about the custom ErrorHandler
> > impl, and what the current DSL
> > would have to look like. I don't have to specify any endpoint for the
> > ErrorHandler right?
>
> The current default error handler uses any-old-processor as the dead
> letter queue; so you could configure separate endpoints depending on a
> system or business exception; but maybe you just wanna change the
> properties (retry count / backoff times etc) and leave the default
> behaviour...
>
>
> > In that case I could just manually post messages to the "business exception"
> > endpoint if the exception type matches using a spring configured proxy, or
> > otherwise just go into an infinite loop. I'll read the source for
> > DefaultErrorHandler to get a better understanding of the ErrorHandlers in
> > general.
>
> I'm even wondering if the DSL should have exception handling as a core
> feature. I guess we can either, add routing/filtering inside a
> specific error handler; or we could add explicit error-handling
> routing constructs.
>
>
> > >Firstly you can customize the errorHandler on a set of routes, or a
> > >specific route, or within the processing of a route. So if you had
> > >your own idea errorHandler implementation you could use it wherever
> > >you like
> >
> > >Currently there's no 'exception-type-based error handler'
> > >implementation; but it should be easy to add such a thing. e.g. we
> > >could add a Set<Class> property to the DeadLetterChannel for
> > >exceptions which should not be retried maybe?
> >
> >
> > > If I could then I suppose I could move business exceptions immediately to
> > > the hospital queue (with a retry count of zero).
> > >
> > > I could also bind a different error handler with an infinite retry count
> > > (-1?).
> >
> > >Yeah - I've been pondering similar things. I'm wondering if we should
> > >have some kinda version of the Content Based Router that uses a
> > >similar notation to try/catch/finally; where known business related
> > >exceptions are caught and the message forwarded to a 'business logic
> > >failed' type endpoint, otherwise system exceptions (which are way more
> > >random and harder to catch all) filter up to the default errorHandler
> > >- such as for retrying & dead letter channels etc.
> >
> > >e.g. in XML I was pondering something like
> >
> > <try>
> >   <!-- do some stuff here which may throw exceptions -->
> >
> >   <catch error="org.foo.bar.SomeBusinessException">
> >     <!-- for business level exceptions, send to some reject queue -->
> >     <to uri="activemq:myRejections'/>
> >   </catch>
> >
> >   <!-- any other system exceptions cause the error handler to retry n-times
> > -->
> > </try>
> >
> > >Thoughts?
> >
> > yes this would also seem to do the trick. Would this style be supported in
> > the Java DSL?
>
> Ideally yes - both Java and XML
>
>
> > I'm just going into the office. I'll be back in an hour or so. Hopefully the
> > DSL above clarifies what we need.
>
> Yes thanks! I've just woke up - a tad early - so need a bit more
> coffee to ponder this a bit more; as I think there's a few ways we
> could go, am currently not sure the best way (or if we should do all
> of them :)
>
> * we could have some kinda metadata registry thing, like the Type
> Converter, to indicate what exceptions are really system exceptions
> versus business exceptions, so that the default error handler can just
> do the right thing (e.g. have default, or customizable retry counts by
> type (business v system) or even customize by a specific exception
> type etc
>
> * we could add routing inside the error handler as you suggest
>
> * we could add some kinda try/catch type construct to the DSL so that
> the DSL itself can describe the routing & handling of exceptions
>
> The last 2 are kinda similar in some ways. I guess each have strengths
> and weaknesses; I kinda like simplifying the DSL as much as possible
> and having things do the right thing without needing much explicit
> coding (rather like the type conversion stuff); but sometimes making
> the DSL totally explicit (e.g. in terms of error handling or type
> conversion) can be useful.
>
> Let me have more coffee & I'll get back to you :)

I've just added to trunk a little try/catch style routing DSL.

In Java we can't use try/catch/finally words so I've initially gone with this

  from("direct:start").
          tryBlock().
              process(validator).
              to("mock:valid").
          handle(ValidationException.class).to("mock:invalid");

which is maybe not the easiest thing to read; but tryBlock / handle is
a replacement for try/catch. (Maybe tryBlock/catchBlock is more
consistent?). Any name suggestions most welcome! (Its often the
hardest part of programming, coming up with good names for things).

In XML its a little easier on the eye. e.g. here's a validation test
case to ensure that messages are delivered to different endpoints
depending on whether they are valid or not...
https://svn.apache.org/repos/asf/activemq/camel/trunk/components/camel-spring/src/test/resources/org/apache/camel/component/validator/camelContext.xml

In Java we could even use the real try/catch handling using an inline Processor

e.g.

	from("direct:start").process(new Processor() {
	    public void process(Exchange exchange) throws Exception {
	        try {
	            validator.process(exchange);

	            template.send("mock:valid", exchange);
	        }
	        catch (ValidationException e) {
	            template.send("mock:invalid", exchange);
	        }
	    }
	});

Thoughts?

-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

On 8/7/07, Neil Thorne <mr...@hotmail.com> wrote:
>
> Hi Neil
> Hi James thanks for the quick reply.
>
> >> a) a system exception gets thrown. A system exception indicates an
> >> infrastructure problem that will be fixed ASAP as it is essential to the
> >> correct functioning of the system as a whole. eg. jdbc connection failed,
> >> remote system is down etc. If a system exception is thrown then that
> >> operation should be continually retried until it is successful.
> >>
> >> b) a business exception is thrown. A business exception indicates that
> >> there
> >> was a problem with the request itself. It should not be retried (at least
> >> not until the problem has been resolved). The message should be moved
> >> onto a
> >> hospital queue where it can be inspected, changed and retried after
> >> possible
> >> manual intervention.
>
> > BTW how can you tell the difference between a) and b)? Are there
> > certain base exception types you can use?
>
> Yes I can. Although our applications can throw quite a few different
> exceptions, so far in terms of routing we only need these two base types. We
> have translators that convert the original exception to one of these types.

Ah cool. I was wondering; rather like the Type Converter stuff....
http://activemq.apache.org/camel/type-converter.html

Do we need a simple little framework where we can easily mark
exception types as being 'system exceptions' or 'business exceptions';
so the error handler can do the right thing by default without
explicit configuration in the DSL?

e.g. for code you own, we could add an @BusinessException or
@SystemException annotation. The annotation could even configure the
retry count/timeout to be specified on a per exception basis.

For code that we don't own, we could add some kinda
META-INF/services/org/apache/camel/BusinessExceptions file that lists
the base classes of all the business level exceptions (or system
exceptions etc).

Am thinking btw that it might be easier to mark just the business
level exceptions; as the system exceptions tend to be more diverse &
harder to enumerate.

If we could know using the 'ExceptionRepository' what exceptions are
which, then we can know whether or not to retry etc.

Am even thinking, maybe we could attach different error handling to
certain types of exception. e.g. if we can ever really detect a true
database deadlock (which is getting harder these days due to exception
wrapping etc) - we might want to increase the retry counts etc.


> >> What is the simplest way to achieve this with camel? Currently I can only
> >> see an errorHandler which assumes you want to dump messages that cannot
> >> be
> >> processed to an error endpoint.
>
> >An ErrorHandler could do anything - e.g. based on the type of
> >exception it could decide whether or not to retry (if at all) etc.
>
> >The default implementation is DeadLetterChannelErrorHandler, which
> >assumes n-retries with optional backoff delays - before sending to a
> >'dead letter' endpoint, which is kinda like your hospital queue
>
> Okay I can write my own implementation.

Please let us know how you get on - as I'd love something like this to
be in the core of Camel.

e.g. maybe we can patch the default DeadLetterChannelErrorHandler to
use some kinda pluggable strategy to determine if an exception is a
business level exception (and so the retry logic is not used etc)


> We're using the Java DSL style of
> configuration.
>
> I'd need to set two different retry strategies depending on the type of
> exception.
>
> from("direct:a").errorHandler().filter(
> type.isEqualTo("com.acme.BusinessException").
> maximumRedeliveries(0).to("direct:business.errors")
> ).errorHandler().filter(
> type.isEqualTo("com.acme.SystemException").
> maximumRedeliveries(-1).useExponentialBackOff()
> ).to("direct:b");

Looks good :)


> Without explicit DSL support I'm wondering about the custom ErrorHandler
> impl, and what the current DSL
> would have to look like. I don't have to specify any endpoint for the
> ErrorHandler right?

The current default error handler uses any-old-processor as the dead
letter queue; so you could configure separate endpoints depending on a
system or business exception; but maybe you just wanna change the
properties (retry count / backoff times etc) and leave the default
behaviour...


> In that case I could just manually post messages to the "business exception"
> endpoint if the exception type matches using a spring configured proxy, or
> otherwise just go into an infinite loop. I'll read the source for
> DefaultErrorHandler to get a better understanding of the ErrorHandlers in
> general.

I'm even wondering if the DSL should have exception handling as a core
feature. I guess we can either, add routing/filtering inside a
specific error handler; or we could add explicit error-handling
routing constructs.


> >Firstly you can customize the errorHandler on a set of routes, or a
> >specific route, or within the processing of a route. So if you had
> >your own idea errorHandler implementation you could use it wherever
> >you like
>
> >Currently there's no 'exception-type-based error handler'
> >implementation; but it should be easy to add such a thing. e.g. we
> >could add a Set<Class> property to the DeadLetterChannel for
> >exceptions which should not be retried maybe?
>
>
> > If I could then I suppose I could move business exceptions immediately to
> > the hospital queue (with a retry count of zero).
> >
> > I could also bind a different error handler with an infinite retry count
> > (-1?).
>
> >Yeah - I've been pondering similar things. I'm wondering if we should
> >have some kinda version of the Content Based Router that uses a
> >similar notation to try/catch/finally; where known business related
> >exceptions are caught and the message forwarded to a 'business logic
> >failed' type endpoint, otherwise system exceptions (which are way more
> >random and harder to catch all) filter up to the default errorHandler
> >- such as for retrying & dead letter channels etc.
>
> >e.g. in XML I was pondering something like
>
> <try>
>   <!-- do some stuff here which may throw exceptions -->
>
>   <catch error="org.foo.bar.SomeBusinessException">
>     <!-- for business level exceptions, send to some reject queue -->
>     <to uri="activemq:myRejections'/>
>   </catch>
>
>   <!-- any other system exceptions cause the error handler to retry n-times
> -->
> </try>
>
> >Thoughts?
>
> yes this would also seem to do the trick. Would this style be supported in
> the Java DSL?

Ideally yes - both Java and XML


> I'm just going into the office. I'll be back in an hour or so. Hopefully the
> DSL above clarifies what we need.

Yes thanks! I've just woke up - a tad early - so need a bit more
coffee to ponder this a bit more; as I think there's a few ways we
could go, am currently not sure the best way (or if we should do all
of them :)

* we could have some kinda metadata registry thing, like the Type
Converter, to indicate what exceptions are really system exceptions
versus business exceptions, so that the default error handler can just
do the right thing (e.g. have default, or customizable retry counts by
type (business v system) or even customize by a specific exception
type etc

* we could add routing inside the error handler as you suggest

* we could add some kinda try/catch type construct to the DSL so that
the DSL itself can describe the routing & handling of exceptions

The last 2 are kinda similar in some ways. I guess each have strengths
and weaknesses; I kinda like simplifying the DSL as much as possible
and having things do the right thing without needing much explicit
coding (rather like the type conversion stuff); but sometimes making
the DSL totally explicit (e.g. in terms of error handling or type
conversion) can be useful.

Let me have more coffee & I'll get back to you :)
-- 
James
-------
http://macstrac.blogspot.com/

Re: Exception based routing

[Neil Thorne <mr...@hotmail.com>]

Hi Neil
Hi James thanks for the quick reply.

>> a) a system exception gets thrown. A system exception indicates an
>> infrastructure problem that will be fixed ASAP as it is essential to the
>> correct functioning of the system as a whole. eg. jdbc connection failed,
>> remote system is down etc. If a system exception is thrown then that
>> operation should be continually retried until it is successful.
>>
>> b) a business exception is thrown. A business exception indicates that
>> there
>> was a problem with the request itself. It should not be retried (at least
>> not until the problem has been resolved). The message should be moved
>> onto a
>> hospital queue where it can be inspected, changed and retried after
>> possible
>> manual intervention.

> BTW how can you tell the difference between a) and b)? Are there
> certain base exception types you can use?

Yes I can. Although our applications can throw quite a few different
exceptions, so far in terms of routing we only need these two base types. We
have translators that convert the original exception to one of these types. 

>> What is the simplest way to achieve this with camel? Currently I can only
>> see an errorHandler which assumes you want to dump messages that cannot
>> be
>> processed to an error endpoint.

>An ErrorHandler could do anything - e.g. based on the type of
>exception it could decide whether or not to retry (if at all) etc.

>The default implementation is DeadLetterChannelErrorHandler, which
>assumes n-retries with optional backoff delays - before sending to a
>'dead letter' endpoint, which is kinda like your hospital queue

Okay I can write my own implementation. We're using the Java DSL style of
configuration.

I'd need to set two different retry strategies depending on the type of
exception.

from("direct:a").errorHandler().filter(
type.isEqualTo("com.acme.BusinessException").
maximumRedeliveries(0).to("direct:business.errors")
).errorHandler().filter(
type.isEqualTo("com.acme.SystemException").
maximumRedeliveries(-1).useExponentialBackOff()
).to("direct:b");   

Without explicit DSL support I'm wondering about the custom ErrorHandler
impl, and what the current DSL
would have to look like. I don't have to specify any endpoint for the
ErrorHandler right?

In that case I could just manually post messages to the "business exception"
endpoint if the exception type matches using a spring configured proxy, or
otherwise just go into an infinite loop. I'll read the source for
DefaultErrorHandler to get a better understanding of the ErrorHandlers in
general.

>Firstly you can customize the errorHandler on a set of routes, or a
>specific route, or within the processing of a route. So if you had
>your own idea errorHandler implementation you could use it wherever
>you like

>Currently there's no 'exception-type-based error handler'
>implementation; but it should be easy to add such a thing. e.g. we
>could add a Set<Class> property to the DeadLetterChannel for
>exceptions which should not be retried maybe?


> If I could then I suppose I could move business exceptions immediately to
> the hospital queue (with a retry count of zero).
>
> I could also bind a different error handler with an infinite retry count
> (-1?).

>Yeah - I've been pondering similar things. I'm wondering if we should
>have some kinda version of the Content Based Router that uses a
>similar notation to try/catch/finally; where known business related
>exceptions are caught and the message forwarded to a 'business logic
>failed' type endpoint, otherwise system exceptions (which are way more
>random and harder to catch all) filter up to the default errorHandler
>- such as for retrying & dead letter channels etc.
 
>e.g. in XML I was pondering something like

<try>
  <!-- do some stuff here which may throw exceptions -->

  <catch error="org.foo.bar.SomeBusinessException">
    <!-- for business level exceptions, send to some reject queue -->
    <to uri="activemq:myRejections'/>
  </catch>

  <!-- any other system exceptions cause the error handler to retry n-times
-->
</try>

>Thoughts?

>-- 
>James
>-------
>http://macstrac.blogspot.com/

yes this would also seem to do the trick. Would this style be supported in
the Java DSL?

I'm just going into the office. I'll be back in an hour or so. Hopefully the
DSL above clarifies what we need.

thanks

Neil

-- 
View this message in context: http://www.nabble.com/Exception-based-routing-tf4226796s22882.html#a12029306
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Exception based routing

[James Strachan <ja...@gmail.com>]

Hi Neil

On 8/6/07, Neil Thorne <mr...@hotmail.com> wrote:
>
> Hi,
>
> I work with Nick Outram - we've a splitter of sorts working (with our own
> processor)
>
> I have the requirement where on a route from A to B there are a couple of
> different exceptions that can be thrown.

Funny - I was just thinking about that today. I've been working on
some XML validation components for XSD/Relax and wanted a nice way to
route based on validation exceptions...


> a) a system exception gets thrown. A system exception indicates an
> infrastructure problem that will be fixed ASAP as it is essential to the
> correct functioning of the system as a whole. eg. jdbc connection failed,
> remote system is down etc. If a system exception is thrown then that
> operation should be continually retried until it is successful.
>
> b) a business exception is thrown. A business exception indicates that there
> was a problem with the request itself. It should not be retried (at least
> not until the problem has been resolved). The message should be moved onto a
> hospital queue where it can be inspected, changed and retried after possible
> manual intervention.

BTW how can you tell the difference between a) and b)? Are there
certain base exception types you can use?


> What is the simplest way to achieve this with camel? Currently I can only
> see an errorHandler which assumes you want to dump messages that cannot be
> processed to an error endpoint.

An ErrorHandler could do anything - e.g. based on the type of
exception it could decide whether or not to retry (if at all) etc.

The default implementation is DeadLetterChannelErrorHandler, which
assumes n-retries with optional backoff delays - before sending to a
'dead letter' endpoint, which is kinda like your hospital queue


>  Can I bind an ErrorHandler to a specific
> type of exception on the same route somehow?

Firstly you can customize the errorHandler on a set of routes, or a
specific route, or within the processing of a route. So if you had
your own idea errorHandler implementation you could use it wherever
you like

Currently there's no 'exception-type-based error handler'
implementation; but it should be easy to add such a thing. e.g. we
could add a Set<Class> property to the DeadLetterChannel for
exceptions which should not be retried maybe?


> If I could then I suppose I could move business exceptions immediately to
> the hospital queue (with a retry count of zero).
>
> I could also bind a different error handler with an infinite retry count
> (-1?).

Yeah - I've been pondering similar things. I'm wondering if we should
have some kinda version of the Content Based Router that uses a
similar notation to try/catch/finally; where known business related
exceptions are caught and the message forwarded to a 'business logic
failed' type endpoint, otherwise system exceptions (which are way more
random and harder to catch all) filter up to the default errorHandler
- such as for retrying & dead letter channels etc.

e.g. in XML I was pondering something like

<try>
  <!-- do some stuff here which may throw exceptions -->

  <catch error="org.foo.bar.SomeBusinessException">
    <!-- for business level exceptions, send to some reject queue -->
    <to uri="activemq:myRejections'/>
  </catch>

  <!-- any other system exceptions cause the error handler to retry n-times -->
</try>

Thoughts?

-- 
James
-------
http://macstrac.blogspot.com/