You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Paul Gale <pa...@gmail.com> on 2014/03/28 18:06:12 UTC
Camel, concurrent consumers, message groups and prefetch size
Hi,
I have an embedded Camel client that contains a single route. All messages
the route consumes have their JMSXGroupID property set.
What I need to know is how message grouping behaves in the context of
concurrent consumers and how does the choice of the prefetch buffer size
come into play (when not using the default)?
Say I have route that looks like:
from("activemq:some_queue?concurrentConsumers=5&maxConcurrentConsumers=5&destination.consumer.prefetchSize=5")
.bean(new SomeBean());
- Is it guaranteed that a given message group will always be handled by the
same consumer thread?
- Is the prefetch size applied at the consumer thread level or to the
single underlying connection that they're all sharing?
- If the prefetch size is applied at the connection level, is it correct to
assume that its value must be >= the number of concurrent consumers to
avoid leaving threads idle when there are messages available to be
processed, example: when consumer.prefetchSize=1?
Yes, I have read this page:
https://camel.apache.org/parallel-processing-and-ordering.html but it
didn't answer my questions.
Thanks,
Paul
Re: Camel, concurrent consumers, message groups and prefetch size
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I suggest to check the AMQ docs, and ask on their user forum as the
questions is more AMQ specific than Camel.
There is also some people who have blogged about AMQ you can find
valuable information form, such as blogs from Torsten Mielke and
Christian Posta.
On Fri, Mar 28, 2014 at 6:06 PM, Paul Gale <pa...@gmail.com> wrote:
> Hi,
>
> I have an embedded Camel client that contains a single route. All messages
> the route consumes have their JMSXGroupID property set.
>
> What I need to know is how message grouping behaves in the context of
> concurrent consumers and how does the choice of the prefetch buffer size
> come into play (when not using the default)?
>
> Say I have route that looks like:
>
> from("activemq:some_queue?concurrentConsumers=5&maxConcurrentConsumers=5&destination.consumer.prefetchSize=5")
> .bean(new SomeBean());
>
> - Is it guaranteed that a given message group will always be handled by the
> same consumer thread?
>
> - Is the prefetch size applied at the consumer thread level or to the
> single underlying connection that they're all sharing?
>
> - If the prefetch size is applied at the connection level, is it correct to
> assume that its value must be >= the number of concurrent consumers to
> avoid leaving threads idle when there are messages available to be
> processed, example: when consumer.prefetchSize=1?
>
> Yes, I have read this page:
> https://camel.apache.org/parallel-processing-and-ordering.html but it
> didn't answer my questions.
>
> Thanks,
> Paul
--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Make your Camel applications look hawt, try: http://hawt.io
Re: Camel, concurrent consumers, message groups and prefetch size
Posted by Paul Gale <pa...@gmail.com>.
claus.takeLeafFrom(christian.getBook());
:)
On Mon, Mar 31, 2014 at 5:09 PM, Christian Posta
<ch...@gmail.com>wrote:
> On Fri, Mar 28, 2014 at 10:06 AM, Paul Gale <pa...@gmail.com> wrote:
> > Hi,
> >
> > I have an embedded Camel client that contains a single route. All
> messages
> > the route consumes have their JMSXGroupID property set.
> >
> > What I need to know is how message grouping behaves in the context of
> > concurrent consumers and how does the choice of the prefetch buffer size
> > come into play (when not using the default)?
>
> It's the same as if not using concurrent consumers, as described on
> the ActiveMQ wiki... however,
> each one of your consumers will compete for its own individual message
> group.
>
> >
> > Say I have route that looks like:
> >
> >
> from("activemq:some_queue?concurrentConsumers=5&maxConcurrentConsumers=5&destination.consumer.prefetchSize=5")
> > .bean(new SomeBean());
> >
> > - Is it guaranteed that a given message group will always be handled by
> the
> > same consumer thread?
> Same logical thread, yes. The same consumer will consume the messages
> for which group it's assigned up until the group closes or the
> consumer is closed.
>
> >
> > - Is the prefetch size applied at the consumer thread level or to the
> > single underlying connection that they're all sharing?
>
> It's applied at the individual consumer, not at the connection.
>
> >
> > - If the prefetch size is applied at the connection level, is it correct
> to
> > assume that its value must be >= the number of concurrent consumers to
> > avoid leaving threads idle when there are messages available to be
> > processed, example: when consumer.prefetchSize=1?
> >
> > Yes, I have read this page:
> > https://camel.apache.org/parallel-processing-and-ordering.html but it
> > didn't answer my questions.
> >
> > Thanks,
> > Paul
>
>
>
> --
> Christian Posta
> http://www.christianposta.com/blog
> twitter: @christianposta
>
Re: Camel, concurrent consumers, message groups and prefetch size
Posted by Christian Posta <ch...@gmail.com>.
On Fri, Mar 28, 2014 at 10:06 AM, Paul Gale <pa...@gmail.com> wrote:
> Hi,
>
> I have an embedded Camel client that contains a single route. All messages
> the route consumes have their JMSXGroupID property set.
>
> What I need to know is how message grouping behaves in the context of
> concurrent consumers and how does the choice of the prefetch buffer size
> come into play (when not using the default)?
It's the same as if not using concurrent consumers, as described on
the ActiveMQ wiki... however,
each one of your consumers will compete for its own individual message group.
>
> Say I have route that looks like:
>
> from("activemq:some_queue?concurrentConsumers=5&maxConcurrentConsumers=5&destination.consumer.prefetchSize=5")
> .bean(new SomeBean());
>
> - Is it guaranteed that a given message group will always be handled by the
> same consumer thread?
Same logical thread, yes. The same consumer will consume the messages
for which group it's assigned up until the group closes or the
consumer is closed.
>
> - Is the prefetch size applied at the consumer thread level or to the
> single underlying connection that they're all sharing?
It's applied at the individual consumer, not at the connection.
>
> - If the prefetch size is applied at the connection level, is it correct to
> assume that its value must be >= the number of concurrent consumers to
> avoid leaving threads idle when there are messages available to be
> processed, example: when consumer.prefetchSize=1?
>
> Yes, I have read this page:
> https://camel.apache.org/parallel-processing-and-ordering.html but it
> didn't answer my questions.
>
> Thanks,
> Paul
--
Christian Posta
http://www.christianposta.com/blog
twitter: @christianposta