Consumer performance

[cdmh <cd...@googlemail.com>]

I am trying to understand the performance profile of ActuveMQ. I have a queue
with 1,000,000 messages. Running a single consumer process (C++ using CMS
MessageListener, auto acknowledge) I get consume 700-800 msg per second. Run
a second and they both run at ~500m/s. The performance degrades for each new
consumer I add, down to ~140m/s with 6 consumers.

I tried a test to dequeue messages with a manual ack before actually
processing the message, but the results were not that different. Posting to
a different queue occurs during the processing, but no posting to the queue
being consumed.

Can anyone give me any pointers on how to fix this issue?

Thanks
-- Craig 

--
View this message in context: http://activemq.2283324.n4.nabble.com/Consumer-performance-tp4653260.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer performance

[cdmh <cd...@googlemail.com>]

Thanks for the pointers, I'll take a look and run some more tests.

I have run tests with consumers on a Linux VM and on the Windows host, where
the ampq server is on the Windows Host too. The machine is 8-core with 8Gb
RAM and the network is (obviously) virtual between VM and host. Messages are
typically 1-4Kb.

--
View this message in context: http://activemq.2283324.n4.nabble.com/Consumer-performance-tp4653260p4653282.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer performance

[mickhayes <mi...@gmail.com>]

I would fiddle with the prefetch limits to begin with. To see if it makes a
difference, you can increase the queue prefetch limit for your consumers as
much as you can (subject to heapspace constraints on the consumers).

Estimate the extra heapspace needed to hold the prefetched messages in the
consumer by

prefetch_limit x message_size = extra 

and make sure you have plenty.

http://activemq.apache.org/what-is-the-prefetch-limit-for.html


You say the performance-per-consumer drops significantly as you add
consumers...
Just to be sure: are your consumers sharing no resources (apart from the
broker on a different box?)

Are your messages reasonably small and your network pipes sufficiently
large?

As well as info on the prefetch limit, there is more tuning information in
ActiveMQ in Action, section 13.3 "Optimizing message consumers"

In particular, you should look at DUPS_OK_ACKNOWLEDGE instead of the auto
ack that you are using.






-----
Michael Hayes B.Sc. (NUI), M.Sc. (DCU), SCSA SCNA 

--
View this message in context: http://activemq.2283324.n4.nabble.com/Consumer-performance-tp4653260p4653278.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.