Why producer to a queue or durable topic consumes a lot of CPU?

[janylj <ja...@gmail.com>]

I had a single broker, which uses the default configuration. Then I had a
Producer sending messages to a queue and a Consumer reading from the queue.
At this time, the CPU usage on the broker is about 60%-70%. Because the
broker is on a box with 2 Xeon processes each @ 1.86GHz, each processor has
more than 50% idle. Then I added another Producer and Consumer to the queue,
it drove the combined CPU usage over 100% and each processor only had 30%
idle. I think mostly Producer is taking CPU resource, adding more Consumer
doesn't seem to put much CPU load.

Finally the message rate for one producer is about 4K message/second. While
the rate when having three producers simultaneously only is 1.5K
message/second each.

The question is that whether it's expected from Activemq or something I did
wrong. Also does it imply that adding more producer does not increase
throughput, if not decrease it?

Thank you very much.
-- 
View this message in context: http://www.nabble.com/Why-producer-to-a-queue-or-durable-topic-consumes-a-lot-of-CPU--tp17524607s2354p17524607.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Why producer to a queue or durable topic consumes a lot of CPU?

[Rob Davies <ra...@gmail.com>]

On 29 May 2008, at 00:57, janylj wrote:

>
> I had a single broker, which uses the default configuration. Then I  
> had a
> Producer sending messages to a queue and a Consumer reading from the  
> queue.
> At this time, the CPU usage on the broker is about 60%-70%. Because  
> the
> broker is on a box with 2 Xeon processes each @ 1.86GHz, each  
> processor has
> more than 50% idle. Then I added another Producer and Consumer to  
> the queue,
> it drove the combined CPU usage over 100% and each processor only  
> had 30%
> idle. I think mostly Producer is taking CPU resource, adding more  
> Consumer
> doesn't seem to put much CPU load.
>
> Finally the message rate for one producer is about 4K message/ 
> second. While
> the rate when having three producers simultaneously only is 1.5K
> message/second each.
>
> The question is that whether it's expected from Activemq or  
> something I did
> wrong. Also does it imply that adding more producer does not increase
> throughput, if not decrease it?
>
> Thank you very much.
> -- 
> View this message in context: http://www.nabble.com/Why-producer-to-a-queue-or-durable-topic-consumes-a-lot-of-CPU--tp17524607s2354p17524607.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

There is a lot of work going on in the broker marshaling/unmarshaling  
messages, adding them to an index in store etc - you will hit a  
plateau where you won't increase overall throughput by adding more  
producers. From your example, the overall throughput has increased  
from 4k msg/sec to 4.5k msg/sec by using 3 producers instead of 1 - so  
you haven't quite reached that point yet - but your close.

We will be looking at increasing throughput and reducing cpu for  
version 6.0 - which will be due out later this year.


cheers,

Rob

http://open.iona.com/products/enterprise-activemq
http://rajdavies.blogspot.com/