Jms endpoint: process messages in parallel, but keeping predefined order

[Alexey Vinogradov <al...@db.com>]

I need to solve the following problem:
There is JMS queue where the messages are consumed from. Each message has
business Id and version. The task is to process messages in parallel, but
messages with the same ID should be processed in the order defined by
version. The processing should be reliable.  It means that if any error
happens while message is being processed the JMS message should not be
acknowledged, to be read again when the error cause is eliminated. No
messages can be lost due to any error.

Example:
channel contains messages: id[1]ver[1], id[2]ver[1], id[3]ver[1],
id[1]ver[2]. First 3 messages can be processed in parallel as Ids are
different, but the 4-th message cannot be processed before the processing of
the 1-st version completes successfully.

There is a custom solution based on bare JMS API. I wonder if there is any
out-of-the box way to solve this task using Camel?


--
View this message in context: http://camel.465427.n5.nabble.com/Jms-endpoint-process-messages-in-parallel-but-keeping-predefined-order-tp4487032p4487032.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Jms endpoint: process messages in parallel, but keeping predefined order

[Alexey Vinogradov <al...@db.com>]

Hi Claus,

These message groups look very similar to the solution with JMS selectors.
Dividing ID's into a number of non intersecting subsets. Creating the same
number of single-thread consumers with selectors will guarantee that the
messages with same ID will be processed sequentially by the same consumer.

It is one of possible solutions. In our case the solution is not flexible
enough due to our existing infrastructure. 

I was looking for something more Camel way. Something like JMS polling
consumer with thread dispatcher, that keeps track of processing threads and
messages being processed in those threads. Anything similar shipped with
Camel?

--
View this message in context: http://camel.465427.n5.nabble.com/Jms-endpoint-process-messages-in-parallel-but-keeping-predefined-order-tp4487032p4487469.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Jms endpoint: process messages in parallel, but keeping predefined order

[Claus Ibsen <cl...@gmail.com>]

Hi

Read about message groups
http://activemq.apache.org/message-groups.html
http://activemq.apache.org/how-do-i-preserve-order-of-messages.html


On Tue, Jun 14, 2011 at 9:46 AM, Alexey Vinogradov
<al...@db.com> wrote:
> I need to solve the following problem:
> There is JMS queue where the messages are consumed from. Each message has
> business Id and version. The task is to process messages in parallel, but
> messages with the same ID should be processed in the order defined by
> version. The processing should be reliable.  It means that if any error
> happens while message is being processed the JMS message should not be
> acknowledged, to be read again when the error cause is eliminated. No
> messages can be lost due to any error.
>
> Example:
> channel contains messages: id[1]ver[1], id[2]ver[1], id[3]ver[1],
> id[1]ver[2]. First 3 messages can be processed in parallel as Ids are
> different, but the 4-th message cannot be processed before the processing of
> the 1-st version completes successfully.
>
> There is a custom solution based on bare JMS API. I wonder if there is any
> out-of-the box way to solve this task using Camel?
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Jms-endpoint-process-messages-in-parallel-but-keeping-predefined-order-tp4487032p4487032.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/