You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Raul Kripalani (Created) (JIRA)" <ji...@apache.org> on 2011/12/09 14:24:39 UTC
[jira] [Created] (CAMEL-4761) Easier, more transparent config for
crash-proof routes
Easier, more transparent config for crash-proof routes
------------------------------------------------------
Key: CAMEL-4761
URL: https://issues.apache.org/jira/browse/CAMEL-4761
Project: Camel
Issue Type: New Feature
Components: camel-core
Affects Versions: 2.8.3, 2.9.0
Reporter: Raul Kripalani
Only applicable to InOnly exchanges.
Using the right combination of Camel route + ActiveMQ or other JMS provider for internal routing, it is straight forward to achieve crash-proof routing. That is, if the container crashes in the middle of routing, when it comes back up it can resume processing from where it cut off. For this, we use the JMS broker as our persistent medium. This allows Camel to be lightweight and to not depend on a DB (like other middleware offerings do).
However, this entails splitting the route in several routes, chained together via JMS queues. For example:
{code}
from("cxf:bean:...")
.to("xslt:...")
.to("cxf:bean:...")
.to("activemq:queue:route.block2");
from("activemq:queue:route.block2")
.to("xslt:...")
.to("cxf:bean:...")
.to("activemq:queue:route.block3");
from("activemq:queue:route.block3")
...;
{code}
Even though the goal is achieved, the syntax is quite verbose and creates queue proliferation in the broker. It complicates the routing logic and is intrusive, since it forces the developer to "pollute" their code with concerns that could be handled by the middleware itself.
*Ideas/paths to explore:*
* Provide a quick and easy way for folks to signal "checkpoints" or "savepoints" and have Camel take care of diverting out to the broker behind the curtains. Create a new DSL to mark checkpoints and have Camel automatically use the context id, route id and a checkpoint name to create JMS queues and transparently weave the to() and from() from the two sections separated by a checkpoint. For example, the route above turns into:
{code}
from("cxf:bean:...")
.to("xslt:...")
.to("cxf:bean:...")
.checkpoint("block2")
.to("xslt:...")
.to("cxf:bean:...")
.checkpoint("block3)"
...;
{code}
* Create a "reliable:" component wrapper to wrap endpoints and provide idempotence and store replies. Upon a second execution of the route after a failed first attempt, the entire route would be replayed from the beginning and when the reliable endpoints are hit, they would skip invoking the real endpoint and return the response received in the first iteration (thanks James Strachan). Would this require a DB?
* Create an attribute on the <route> element (e.g. "recoverable"), which automatically applies an InterceptionStrategy to divert in and out of the JMS broker before endpoints.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CAMEL-4761) Easier, more transparent config for
crash-proof routes
Posted by "Raul Kripalani (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166160#comment-13166160 ]
Raul Kripalani commented on CAMEL-4761:
---------------------------------------
Attached an example of a route based on Camel 2.8.3 that uses a ping-pong routing style to achieve reliability [https://issues.apache.org/jira/secure/attachment/12506738/test-crashProofProcessing.xml].
The entire route is implemented as a content-based router based on a StepPoint header. The onCompletion block takes care of incrementing the value of header each time a block finishes and returning the message to the JMS broker.
>From there, it is read again and the whole cycle repeats until the last block sets the FinalStep property on the Exchange, which signals onCompletion to stop the ping-pong there.
In this pattern, each when is a processing block pertaining to a sequence. Between each processing block, persistence is provided automatically by the JMS broker and the user doesn't need to worry about sending to the JMS broker themselves.
> Easier, more transparent config for crash-proof routes
> ------------------------------------------------------
>
> Key: CAMEL-4761
> URL: https://issues.apache.org/jira/browse/CAMEL-4761
> Project: Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.8.3, 2.9.0
> Reporter: Raul Kripalani
> Attachments: test-crashProofProcessing.xml
>
>
> Only applicable to InOnly exchanges.
> Using the right combination of Camel route + ActiveMQ or other JMS provider for internal routing, it is straight forward to achieve crash-proof routing. That is, if the container crashes in the middle of routing, when it comes back up it can resume processing from where it cut off. For this, we use the JMS broker as our persistent medium. This allows Camel to be lightweight and to not depend on a DB (like other middleware offerings do).
> However, this entails splitting the route in several routes, chained together via JMS queues. For example:
> {code}
> from("cxf:bean:...")
> .to("xslt:...")
> .to("cxf:bean:...")
> .to("activemq:queue:route.block2");
> from("activemq:queue:route.block2")
> .to("xslt:...")
> .to("cxf:bean:...")
> .to("activemq:queue:route.block3");
> from("activemq:queue:route.block3")
> ...;
> {code}
> Even though the goal is achieved, the syntax is quite verbose and creates queue proliferation in the broker. It complicates the routing logic and is intrusive, since it forces the developer to "pollute" their code with concerns that could be handled by the middleware itself.
> *Ideas/paths to explore:*
> * Provide a quick and easy way for folks to signal "checkpoints" or "savepoints" and have Camel take care of diverting out to the broker behind the curtains. Create a new DSL to mark checkpoints and have Camel automatically use the context id, route id and a checkpoint name to create JMS queues and transparently weave the to() and from() from the two sections separated by a checkpoint. For example, the route above turns into:
> {code}
> from("cxf:bean:...")
> .to("xslt:...")
> .to("cxf:bean:...")
> .checkpoint("block2")
> .to("xslt:...")
> .to("cxf:bean:...")
> .checkpoint("block3)"
> ...;
> {code}
> * Create a "reliable:" component wrapper to wrap endpoints and provide idempotence and store replies. Upon a second execution of the route after a failed first attempt, the entire route would be replayed from the beginning and when the reliable endpoints are hit, they would skip invoking the real endpoint and return the response received in the first iteration (thanks James Strachan). Would this require a DB?
> * Create an attribute on the <route> element (e.g. "recoverable"), which automatically applies an InterceptionStrategy to divert in and out of the JMS broker before endpoints.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CAMEL-4761) Easier, more transparent config for
crash-proof routes
Posted by "Raul Kripalani (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CAMEL-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raul Kripalani updated CAMEL-4761:
----------------------------------
Attachment: test-crashProofProcessing.xml
> Easier, more transparent config for crash-proof routes
> ------------------------------------------------------
>
> Key: CAMEL-4761
> URL: https://issues.apache.org/jira/browse/CAMEL-4761
> Project: Camel
> Issue Type: New Feature
> Components: camel-core
> Affects Versions: 2.8.3, 2.9.0
> Reporter: Raul Kripalani
> Attachments: test-crashProofProcessing.xml
>
>
> Only applicable to InOnly exchanges.
> Using the right combination of Camel route + ActiveMQ or other JMS provider for internal routing, it is straight forward to achieve crash-proof routing. That is, if the container crashes in the middle of routing, when it comes back up it can resume processing from where it cut off. For this, we use the JMS broker as our persistent medium. This allows Camel to be lightweight and to not depend on a DB (like other middleware offerings do).
> However, this entails splitting the route in several routes, chained together via JMS queues. For example:
> {code}
> from("cxf:bean:...")
> .to("xslt:...")
> .to("cxf:bean:...")
> .to("activemq:queue:route.block2");
> from("activemq:queue:route.block2")
> .to("xslt:...")
> .to("cxf:bean:...")
> .to("activemq:queue:route.block3");
> from("activemq:queue:route.block3")
> ...;
> {code}
> Even though the goal is achieved, the syntax is quite verbose and creates queue proliferation in the broker. It complicates the routing logic and is intrusive, since it forces the developer to "pollute" their code with concerns that could be handled by the middleware itself.
> *Ideas/paths to explore:*
> * Provide a quick and easy way for folks to signal "checkpoints" or "savepoints" and have Camel take care of diverting out to the broker behind the curtains. Create a new DSL to mark checkpoints and have Camel automatically use the context id, route id and a checkpoint name to create JMS queues and transparently weave the to() and from() from the two sections separated by a checkpoint. For example, the route above turns into:
> {code}
> from("cxf:bean:...")
> .to("xslt:...")
> .to("cxf:bean:...")
> .checkpoint("block2")
> .to("xslt:...")
> .to("cxf:bean:...")
> .checkpoint("block3)"
> ...;
> {code}
> * Create a "reliable:" component wrapper to wrap endpoints and provide idempotence and store replies. Upon a second execution of the route after a failed first attempt, the entire route would be replayed from the beginning and when the reliable endpoints are hit, they would skip invoking the real endpoint and return the response received in the first iteration (thanks James Strachan). Would this require a DB?
> * Create an attribute on the <route> element (e.g. "recoverable"), which automatically applies an InterceptionStrategy to divert in and out of the JMS broker before endpoints.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira