You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Christian Mueller <ch...@gmail.com> on 2010/06/04 13:20:54 UTC

good solution to work with jms message priority

Hello list,

I have a requirement to process online (coming from an external web service)
and batch (several thousand) messages. The online messages are more prior
(the customer is waiting of the response) and should be processed before the
batch messages (I think a common requirement). Because ActiveMQ at this time
doesn't support message priority
([how-can-i-support-priority-queues|http://activemq.apache.org/how-can-i-support-priority-queues.html]),
I was looking for another solution. I read this thread
[Camel-and-jms-priority|http://old.nabble.com/Camel-and-jms-priority-ts25962858.html#a25962858]
and found it to complicated - I'm lasy... :-)

My question is, would this Camel route also do a good job for this
requirements?

{code}
from("activemq://queue:inbox?concurrentConsumers=5&selector=priority<='4'")
.to("direct:inbox");

from("activemq://queue:inbox?concurrentConsumers=5&selector=priority>'4'")
.to("direct:inbox");

from("direct:inbox")
.processRef("myProcessor")
.to("activemq://queue:outbox");
{code}

Thanks in advanced,
Christian
-- 
View this message in context: http://old.nabble.com/good-solution-to-work-with-jms-message-priority-tp28778639p28778639.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: good solution to work with jms message priority

Posted by Christian Mueller <ch...@gmail.com>.
Thanks for the quick responses...

Mixed processing of messages with different priorities is not a problem for
us. What we have to prevent is, that an online request is only processed
(with a huge delay) after all batch messages (several thousands) are
processed.
Writing in different queues is possible. Than, my JMS consumer route would
looks like:
{code}
from("activemq://queue:inbox.online?concurrentConsumers=5")
.to("direct:inbox");

from("activemq://queue:inbox.batch?concurrentConsumers=5")
.to("direct:inbox");

from("direct:inbox")
.processRef("myProcessor")
.to("activemq://queue:outbox");
{code}

Thanks,
Christian
-- 
View this message in context: http://old.nabble.com/good-solution-to-work-with-jms-message-priority-tp28778639p28779260.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Re: good solution to work with jms message priority

Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jun 4, 2010 at 1:47 PM, Christian Schneider
<ch...@die-schneider.net> wrote:
> How about simply using two separate queues. This way you don´t have to care
> about selectors.
>
> If the sender can not do this then you could still use a content based
> router to route to two queues. Then at least the incoming queue will always
> be emptied quite fast.
> Of course then the gain will depend on how slow your processor is in
> comparisson with the jms messaging.
>

Yeah a good solution when you have control of the message broker and
can create new queues for this purpose.
It would be the best choice as the message can be transacted and
persisted and you wont have issue if the server crashes (well of
course a bit but not as big as having the message lost in the Camel
resequencer)


> Greetings
>
> Christian
>
>
> Am 04.06.2010 13:20, schrieb Christian Mueller:
>>
>> Hello list,
>>
>> I have a requirement to process online (coming from an external web
>> service)
>> and batch (several thousand) messages. The online messages are more prior
>> (the customer is waiting of the response) and should be processed before
>> the
>> batch messages (I think a common requirement). Because ActiveMQ at this
>> time
>> doesn't support message priority
>>
>> ([how-can-i-support-priority-queues|http://activemq.apache.org/how-can-i-support-priority-queues.html]),
>> I was looking for another solution. I read this thread
>>
>> [Camel-and-jms-priority|http://old.nabble.com/Camel-and-jms-priority-ts25962858.html#a25962858]
>> and found it to complicated - I'm lasy... :-)
>>
>> My question is, would this Camel route also do a good job for this
>> requirements?
>>
>> {code}
>>
>> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority<='4'")
>> .to("direct:inbox");
>>
>> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority>'4'")
>> .to("direct:inbox");
>>
>> from("direct:inbox")
>> .processRef("myProcessor")
>> .to("activemq://queue:outbox");
>> {code}
>>
>> Thanks in advanced,
>> Christian
>>
>
> --
> ----
> http://www.liquid-reality.de
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: good solution to work with jms message priority

Posted by Christian Schneider <ch...@die-schneider.net>.
How about simply using two separate queues. This way you don´t have to 
care about selectors.

If the sender can not do this then you could still use a content based 
router to route to two queues. Then at least the incoming queue will 
always be emptied quite fast.
Of course then the gain will depend on how slow your processor is in 
comparisson with the jms messaging.

Greetings

Christian


Am 04.06.2010 13:20, schrieb Christian Mueller:
> Hello list,
>
> I have a requirement to process online (coming from an external web service)
> and batch (several thousand) messages. The online messages are more prior
> (the customer is waiting of the response) and should be processed before the
> batch messages (I think a common requirement). Because ActiveMQ at this time
> doesn't support message priority
> ([how-can-i-support-priority-queues|http://activemq.apache.org/how-can-i-support-priority-queues.html]),
> I was looking for another solution. I read this thread
> [Camel-and-jms-priority|http://old.nabble.com/Camel-and-jms-priority-ts25962858.html#a25962858]
> and found it to complicated - I'm lasy... :-)
>
> My question is, would this Camel route also do a good job for this
> requirements?
>
> {code}
> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority<='4'")
> .to("direct:inbox");
>
> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority>'4'")
> .to("direct:inbox");
>
> from("direct:inbox")
> .processRef("myProcessor")
> .to("activemq://queue:outbox");
> {code}
>
> Thanks in advanced,
> Christian
>    

-- 
----
http://www.liquid-reality.de


Re: good solution to work with jms message priority

Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jun 4, 2010 at 1:20 PM, Christian Mueller
<ch...@gmail.com> wrote:
>
> Hello list,
>
> I have a requirement to process online (coming from an external web service)
> and batch (several thousand) messages. The online messages are more prior
> (the customer is waiting of the response) and should be processed before the
> batch messages (I think a common requirement). Because ActiveMQ at this time
> doesn't support message priority

Yeah I think the AMQ people are looking into this for AMQ 6.

> ([how-can-i-support-priority-queues|http://activemq.apache.org/how-can-i-support-priority-queues.html]),
> I was looking for another solution. I read this thread
> [Camel-and-jms-priority|http://old.nabble.com/Camel-and-jms-priority-ts25962858.html#a25962858]
> and found it to complicated - I'm lasy... :-)
>
> My question is, would this Camel route also do a good job for this
> requirements?
>
> {code}
> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority<='4'")
> .to("direct:inbox");
>
> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority>'4'")
> .to("direct:inbox");
>
> from("direct:inbox")
> .processRef("myProcessor")
> .to("activemq://queue:outbox");
> {code}
>

I think JMS selectors is generally slow. But I assume they can be
used. I wonder through what the statement should be for the priority.
Most likely selector=JMSPriority > 4


However there is a ticket in Camel to enhance the resequencer EIP to
better support JMSPriority so it can batch out messages reordered by
priority
https://issues.apache.org/activemq/browse/CAMEL-2537
I would assume you could try that approach. And then we can improve
the resequencer in Camel so you dont need that "hack" as shown in the
ticket.


Another idea is to use a JDK priority queue and have message from JMS
go to that queue and then have worker threads dequeue from that
priority queue.
In fact we could add such a feature to the Camel SEDA queue, or
something like that.

However in all situations the messages is then retrained in memory and
will be lost if the server crash.
In the future the Camel resequencer will support persistence just as
we did in 2.3 with the aggregator.




> Thanks in advanced,
> Christian
> --
> View this message in context: http://old.nabble.com/good-solution-to-work-with-jms-message-priority-tp28778639p28778639.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Re: good solution to work with jms message priority

Posted by Charles Moulliard <cm...@gmail.com>.
Hi Christian,

As the two routes will create JMS concurrent consumers on the queue
'queue:inbox' with the same consumer.priority, I don't think that
messages with priority > 4 will be processed before the others as the
messages created by camel during the exchange will be send to the same
endpoint "direct:inbox" and then we will have a mix between messages
with different priority.

As mentioned by Claus, EIP resequencer is the only scenario
interesting and selector are very slow on ActiveMQ.

KR,

Charles Moulliard

Senior Enterprise Architect (J2EE, .NET, SOA)
Apache Camel/ServiceMix Committer

*******************************************************************
- Blog : http://cmoulliard.blogspot.com
- Twitter : http://twitter.com/cmoulliard
- Linkedlin : http://www.linkedin.com/in/charlesmoulliard



On Fri, Jun 4, 2010 at 1:20 PM, Christian Mueller
<ch...@gmail.com> wrote:
>
> Hello list,
>
> I have a requirement to process online (coming from an external web service)
> and batch (several thousand) messages. The online messages are more prior
> (the customer is waiting of the response) and should be processed before the
> batch messages (I think a common requirement). Because ActiveMQ at this time
> doesn't support message priority
> ([how-can-i-support-priority-queues|http://activemq.apache.org/how-can-i-support-priority-queues.html]),
> I was looking for another solution. I read this thread
> [Camel-and-jms-priority|http://old.nabble.com/Camel-and-jms-priority-ts25962858.html#a25962858]
> and found it to complicated - I'm lasy... :-)
>
> My question is, would this Camel route also do a good job for this
> requirements?
>
> {code}
> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority<='4'")
> .to("direct:inbox");
>
> from("activemq://queue:inbox?concurrentConsumers=5&selector=priority>'4'")
> .to("direct:inbox");
>
> from("direct:inbox")
> .processRef("myProcessor")
> .to("activemq://queue:outbox");
> {code}
>
> Thanks in advanced,
> Christian
> --
> View this message in context: http://old.nabble.com/good-solution-to-work-with-jms-message-priority-tp28778639p28778639.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>