You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by zaoliu <za...@gmail.com> on 2008/02/22 00:03:35 UTC
Confusing results for my performance testing
An issue when testing performance s when I increase the number of consumers
for testing non-persistent messages, the throughput get down very fast. I
can't find the reason for it. Each consumer thread is a separate connection
to the broker.
Below is my result for testing (all using one producer to send messages in a
separate JVM):
1 consumer: 12683/s
2 consumers: 11289/s
3 consumers: 9956/s
4 consumers: 8638/s
10 consumers: 5820/s
To my understanding, increasing the consumer numbers should improve the
throughput of brokers, but the result is opposite to my expectation.
Zao
--
View this message in context: http://www.nabble.com/Confusing-results-for-my-performance-testing-tp15622219s2354p15622219.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Confusing results for my performance testing
Posted by Zao Liu <za...@cs.indiana.edu>.
Yeah. Broker, consumer and producer are all in separate machine. All these
machines has 6-cpus.
The result for my throughput is total, not count for single consumer that is
why I feel strange of the results.
I also do some tests by using cluster of brokers, typically 3 broker
machines by using store and forward model,
these problem insists.
Sorry, I don't know "top and sar" mean and can you explain in details how
you solve the problem of CPU bounding?
Thanks so much,
Zao
On Thu, Feb 21, 2008 at 3:33 PM, Sean Bastille <se...@gmail.com>
wrote:
> Was this all running on a single 6-cpu host?
>
> I found your initial results surprising given that I have done similar
> testing just a few weeks ago that struggled to make a third of your
> throughput, but my tests involved two 2-cpu hosts, so this likely accounts
> for the difference.
>
> Also to clarify, when you say 10 consumers processed 5820/s, are you
> referring to the total throughput as being 5820, or each consumer handling
> 5820, for a total of 58k/s?
>
> Assuming the former, I believe you are experiencing the same limitations I
> was running into, which is the consumer being CPU bound. You should be
> able
> to investigate further using top or sar.
>
> My peak throughput configuration involved 3 producers and 2 consumers,
> although this was with each process running its own embedded broker.
>
> Sean
>
> On Thu, Feb 21, 2008 at 6:03 PM, zaoliu <za...@gmail.com> wrote:
>
> >
> > An issue when testing performance s when I increase the number of
> > consumers
> > for testing non-persistent messages, the throughput get down very fast.
> I
> > can't find the reason for it. Each consumer thread is a separate
> > connection
> > to the broker.
> > Below is my result for testing (all using one producer to send messages
> in
> > a
> > separate JVM):
> > 1 consumer: 12683/s
> > 2 consumers: 11289/s
> > 3 consumers: 9956/s
> > 4 consumers: 8638/s
> > 10 consumers: 5820/s
> >
> > To my understanding, increasing the consumer numbers should improve the
> > throughput of brokers, but the result is opposite to my expectation.
> >
> > Zao
> > --
> > View this message in context:
> >
> http://www.nabble.com/Confusing-results-for-my-performance-testing-tp15622219s2354p15622219.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
> >
>
Re: Confusing results for my performance testing
Posted by Sean Bastille <se...@gmail.com>.
Was this all running on a single 6-cpu host?
I found your initial results surprising given that I have done similar
testing just a few weeks ago that struggled to make a third of your
throughput, but my tests involved two 2-cpu hosts, so this likely accounts
for the difference.
Also to clarify, when you say 10 consumers processed 5820/s, are you
referring to the total throughput as being 5820, or each consumer handling
5820, for a total of 58k/s?
Assuming the former, I believe you are experiencing the same limitations I
was running into, which is the consumer being CPU bound. You should be able
to investigate further using top or sar.
My peak throughput configuration involved 3 producers and 2 consumers,
although this was with each process running its own embedded broker.
Sean
On Thu, Feb 21, 2008 at 6:03 PM, zaoliu <za...@gmail.com> wrote:
>
> An issue when testing performance s when I increase the number of
> consumers
> for testing non-persistent messages, the throughput get down very fast. I
> can't find the reason for it. Each consumer thread is a separate
> connection
> to the broker.
> Below is my result for testing (all using one producer to send messages in
> a
> separate JVM):
> 1 consumer: 12683/s
> 2 consumers: 11289/s
> 3 consumers: 9956/s
> 4 consumers: 8638/s
> 10 consumers: 5820/s
>
> To my understanding, increasing the consumer numbers should improve the
> throughput of brokers, but the result is opposite to my expectation.
>
> Zao
> --
> View this message in context:
> http://www.nabble.com/Confusing-results-for-my-performance-testing-tp15622219s2354p15622219.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: Confusing results for my performance testing
Posted by Rainer Klute <ra...@dp-itsolutions.de>.
zaoliu schrieb:
> An issue when testing performance s when I increase the number of consumers
> for testing non-persistent messages, the throughput get down very fast. I
> can't find the reason for it. Each consumer thread is a separate connection
> to the broker.
> Below is my result for testing (all using one producer to send messages in a
> separate JVM):
> 1 consumer: 12683/s
> 2 consumers: 11289/s
> 3 consumers: 9956/s
> 4 consumers: 8638/s
> 10 consumers: 5820/s
>
> To my understanding, increasing the consumer numbers should improve the
> throughput of brokers, but the result is opposite to my expectation.
>
To my understanding, increasing the number of consumers should not
change the *overall* throughput. In fact, I'd expect a slightly worse
throughput because the broker has to do more management work than with
just one consumer. However, you point out a serious problem here. See
also <https://issues.apache.org/activemq/browse/AMQ-1586>.
Mit freundlichen Grüßen / Best regards
Rainer Klute
Architect
IT Services Europe
Deutsche Post IT Services GmbH
Konrad-Adenauer-Platz 1
40210 Düsseldorf
Germany
Phone: +49 (0)211 163 1156
Mobile: +49 (0)172 2324824
http://www.dhl.com/
rainer.klute@dp-itsolutions.de
OpenPGP fingerprint: 10F3E02A46E48B5BE7F1B70263F50FB920B926C2