Default pre-fetch behavior of ActiveMQ and the "competing consumer"

[petergdoyle <pe...@comcast.net>]

So I don't want to repeat a Thread Discussion (
http://activemq.2283324.n4.nabble.com/slow-consumer-not-working-td2346908.html
<http://activemq.2283324.n4.nabble.com/slow-consumer-not-working-td2346908.html> 
) but I am not sure the questions were answered about how to control
"prefetch" on consumers using the API. And the Javadocs do not seem to
provide any details on how to use the "prefetch" controls 
http://activemq.apache.org/maven/apidocs/index.html?org/apache/activemq/ActiveMQPrefetchPolicy.html
<http://activemq.apache.org/maven/apidocs/index.html?org/apache/activemq/ActiveMQPrefetchPolicy.html>  
I do not believe the default behavior of ActiveMQ follows the "competing
consumer" model as indicated http://activemq.apache.org/clustering.html.If I
want to make use of an entire compute cluster (message consumers) most
efficiently, I would want the fastest consumers to take up the slack if
slower consumers are not able to keep up. The slow consumer can be cause by
a variety of reasons but it could be nothing more than a mixture of hardware
varying in speed. When I run a test-harness that introduces latency into 1
of 3 "competing" consumers and I send 99 messages from a producer at an even
rate, I see that each of the 3 consumers ingests 33 messages. The 1 that is
sleeping between message handling continues to chug away at its load and the
others sit idle. Even when I stop a fast consumer and restart it, the slow
one still chugs away on his even portion of the total. If I run the same
test-harness with IBM MQ and the same JMS producer / consumer code. I see
the slow consumer take what he can and the balance of the load be evenly
distributed across the fast consumers - as I would expect.I cannot see any
use where the default ActiveMQ behavior is desirable. Regardless, it does
seem to be controlled by a "prefetch" limit. It seems like one could control
the default for all use cases with the setAll() ?void	setAll(int i) void
setDurableTopicPrefetch(int durableTopicPrefetch) void
setMaximumPendingMessageLimit(int maximumPendingMessageLimit)Sets how many
messages a broker will keep around, above the prefetch limit, for
non-durable topics before starting to discard older messages.void
setOptimizeDurableTopicPrefetch(int optimizeAcknowledgePrefetch) void
setQueueBrowserPrefetch(int queueBrowserPrefetch) void	setQueuePrefetch(int
queuePrefetch) void	setTopicPrefetch(int topicPrefetch) So any guidance on
using the ActiveMQPrefetchPolicy class in general is appreciated - as well
while I am assuming I can set the default to allow competing customers to
work at their combined optimal compute capacity and there is a setting for
that... Are there any pitfalls as well?Thank you 



--
View this message in context: http://activemq.2283324.n4.nabble.com/Default-pre-fetch-behavior-of-ActiveMQ-and-the-competing-consumer-tp4712108.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.