You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Sebastien Pennec <se...@qos.ch> on 2007/01/15 17:49:41 UTC
Topic and Queue performances
Hello,
I checked on the list archive, but couldn't find the answer to my current situation,
so here I am :)
I've launched an ActiveMQ server on my machine, with the default configuration stuff.
The only thing I've added is the jndi.properties file that I've found in the
documentation.
This jndi config file helps create an example Queue and an example Topic. I can
easily connect message producers and consumers to the Queue and Topic. The server, as
well as the two message producers and the two message consumers, are on my local machine.
When I send messages to the Topic, I observe that the time it takes to send the
message is about 0.25 milliseconds. When I send messages to the Queue, the figure
goes up to 10 milliseconds. These figures are the average time that is observed when
sending 10'000 messages.
They are modified by the presence or absence of a consumer: it takes approximately
10% less time when there is no consumer.
Is it a normal behavior? I expected the performance between Topic and Queue to be
closer than that.
Thanks for your help,
Sébastien
--
Sébastien Pennec
sebastien@qos.ch
Logback: The reliable, generic, fast and flexible logging framework for Java.
http://logback.qos.ch/
Re: Topic and Queue performances
Posted by Sebastien Pennec <se...@qos.ch>.
Hello Rob,
Then that is certainly why I was observing such different results. Although my setup
(client code and environment) where very similar, the messages were handled in a
rather different manner on the server side.
I've tried to disable persistence by using the persistant="false" attribute in the
broker element. I now have Queue results that are very close to the Topic results, at
around 0.25 milliseconds.
James and Rob, thanks for your help :)
Cheers,
Sébastien
Rob Davies wrote:
> The big difference between topics and queues is that for queues, if the
> messages have a persistent delivery mode, they are always added to the
> message store. This is only true for topics if a durable subscriber
> exists and is interested in that topic.
>
> By default, the message store used is the journal, which uses a jdbc
> store for its long term storage.
>
> cheers,
>
> Rob
>
> On 15 Jan 2007, at 19:39, James Strachan wrote:
>
>> Are you positive you are using like for like - i.e. persistent
>> messages for both queues and topics?
>>
>> You might also wanna try out using our maven based performance test
>> harness which makes it easy to test out your environment - it might be
>> worth seeing if your JMS code gets wildly different results which
>> could indicate some issue in how you are using JMS...
>>
>> http://incubator.apache.org/activemq/activemq-performance-module-users-manual.html
>>
>>
>>
>> On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
>>> Hello James,
>>>
>>> I understand very well that there are many factors that influence
>>> such tests. In my
>>> setup, I've tried to make the environment for both Queue and Topic as
>>> similar as
>>> possible, not to have the best results possible but to have results
>>> that can be compared.
>>>
>>> As far as I know, the default config for Active MQ does not use JDBC,
>>> or does it?
>>> Also, on my tests, it's the Queue that is 40 times slower that the
>>> Topic.
>>>
>>> My goal was more to have your opinion on the fact that, using the
>>> same environment,
>>> sending messages to a Queue is *much* slower that sending to a Topic.
>>> Does that make
>>> you thing that I've done something wrong, or is it something that can
>>> be expected?
>>>
>>> I am currently checking out the source code of ActiveMQ to run some
>>> tests as
>>> described on your website[1] so I'll see how far my figures are from
>>> your tests.
>>>
>>> Thanks for the answer,
>>>
>>> Sébastien
>>>
>>> [1] http://incubator.apache.org/activemq/jmeter-performance-tests.html
>>>
>>>
>>> James Strachan wrote:
>>> > In general, it all depends
>>> >
>>> > http://incubator.apache.org/activemq/how-fast-is-activemq.html
>>> >
>>> > if using JDBC persistence, there is a bit more work required to store
>>> > a topic message than a queue (since topics need to maintain a list of
>>> > messages each consumer requires, whereas a queue is logically
>>> > equivalent to a single row insert). Also obviously, if there are more
>>> > than one consumer, then the broker has more work to do with topics.
>>> >
>>> > But like many things, it all depends on what you are doing, how you're
>>> > doing it and what your environment is. YMMV
>>> >
>>> >
>>> > On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
>>> >> Hello,
>>> >>
>>> >> I checked on the list archive, but couldn't find the answer to my
>>> >> current situation,
>>> >> so here I am :)
>>> >>
>>> >> I've launched an ActiveMQ server on my machine, with the default
>>> >> configuration stuff.
>>> >> The only thing I've added is the jndi.properties file that I've found
>>> >> in the
>>> >> documentation.
>>> >>
>>> >> This jndi config file helps create an example Queue and an example
>>> >> Topic. I can
>>> >> easily connect message producers and consumers to the Queue and
>>> Topic.
>>> >> The server, as
>>> >> well as the two message producers and the two message consumers, are
>>> >> on my local machine.
>>> >>
>>> >> When I send messages to the Topic, I observe that the time it
>>> takes to
>>> >> send the
>>> >> message is about 0.25 milliseconds. When I send messages to the
>>> Queue,
>>> >> the figure
>>> >> goes up to 10 milliseconds. These figures are the average time
>>> that is
>>> >> observed when
>>> >> sending 10'000 messages.
>>> >> They are modified by the presence or absence of a consumer: it takes
>>> >> approximately
>>> >> 10% less time when there is no consumer.
>>> >>
>>> >> Is it a normal behavior? I expected the performance between Topic and
>>> >> Queue to be
>>> >> closer than that.
>>> >>
>>> >> Thanks for your help,
>>> >>
>>> >> Sébastien
>>> >>
>>> >> --
>>> >> Sébastien Pennec
>>> >> sebastien@qos.ch
>>> >>
>>> >> Logback: The reliable, generic, fast and flexible logging framework
>>> >> for Java.
>>> >> http://logback.qos.ch/
>>> >>
>>> >
>>> >
>>>
>>> --
>>> Sébastien Pennec
>>> sebastien@qos.ch
>>>
>>> Logback: The reliable, generic, fast and flexible logging framework
>>> for Java.
>>> http://logback.qos.ch/
>>>
>>
>>
>> --
>> James
>> -------
>> http://radio.weblogs.com/0112098/
>
>
--
Sébastien Pennec
sebastien@qos.ch
Logback: The reliable, generic, fast and flexible logging framework for Java.
http://logback.qos.ch/
Re: Topic and Queue performances
Posted by James Strachan <ja...@gmail.com>.
On 1/15/07, Rainer Klute <kl...@rainer-klute.de> wrote:
> Rob Davies schrieb:
> > The big difference between topics and queues is that for queues, if
> > the messages have a persistent delivery mode, they are always added to
> > the message store. This is only true for topics if a durable
> > subscriber exists and is interested in that topic.
>
> Which is not good. If many messages are added to the topic without being
> taken off from the broker by consumers, the memory is soaked up quickly,
Sure - there's a few different solutions for this, plus you can
increase the RAM buffer with the <usageManager>..
.
http://incubator.apache.org/activemq/slow-consumer-handling.html
> OutOfMemoryErrors occur, and the whole application gets corrupted.
Well it hangs producers until consumers consume stuff to free up space.
> If
> the topic is too full, ActiveMQ should start to swap messages to
> external memory (disk).
It does :) Though you'll need the latest 4.2-SNAPSHOT to get spooling
to disk for non-persistent messaging.
--
James
-------
http://radio.weblogs.com/0112098/
Re: Topic and Queue performances
Posted by Rainer Klute <kl...@rainer-klute.de>.
Rob Davies schrieb:
> that's our bad - we need to post a newer snapshot - at lot of work
> happened over the holidays - and is still ongoing.
I am looking forward for testing any new stuff. For now I helped myself
with a durable subscriber, but that is only a crutch. I am currently
evaluating several JMS providers for being integrated into a larger
project with a few hundred installations over the next years.
> But having said that, it would help us tremendously if folks in
> general could try post a junit test case of a problem - it will ensure
> we can fix a lot quicker.
Will try do so with any future problems.
Best regards
Rainer Klute
Rainer Klute IT-Consulting GmbH
Dipl.-Inform.
Rainer Klute E-Mail: klute@rainer-klute.de
Körner Grund 24 Telefon: +49 172 2324824
D-44143 Dortmund Telefax: +49 231 5349423
OpenPGP fingerprint: E4E4386515EE0BED5C162FBB5343461584B5A42E
Re: Topic and Queue performances
Posted by Rob Davies <ra...@gmail.com>.
that's our bad - we need to post a newer snapshot - at lot of work
happened over the holidays - and is still ongoing.
But having said that, it would help us tremendously if folks in
general could try post a junit test case of a problem - it will
ensure we can fix a lot quicker.
best regards,
Rob Davies
On 16 Jan 2007, at 04:47, Rainer Klute wrote:
> Rob Davies schrieb:
>> Messages are swapped to disk in version 4.2
>
> That's good news! What do I have to do to activate it? I tried a topic
> with non-durable subscribers with ActiveMQ
> 4.2-incubator-20061204.023752-14 and ran into that OutOfMemoryError
> problem.
>
> Best regards
> Rainer Klute
>
> Rainer Klute IT-Consulting GmbH
> Dipl.-Inform.
> Rainer Klute E-Mail: klute@rainer-klute.de
> Körner Grund 24 Telefon: +49 172 2324824
> D-44143 Dortmund Telefax: +49 231 5349423
>
> OpenPGP fingerprint: E4E4386515EE0BED5C162FBB5343461584B5A42E
>
>
Re: Topic and Queue performances
Posted by Rainer Klute <kl...@rainer-klute.de>.
Rob Davies schrieb:
> Messages are swapped to disk in version 4.2
That's good news! What do I have to do to activate it? I tried a topic
with non-durable subscribers with ActiveMQ
4.2-incubator-20061204.023752-14 and ran into that OutOfMemoryError problem.
Best regards
Rainer Klute
Rainer Klute IT-Consulting GmbH
Dipl.-Inform.
Rainer Klute E-Mail: klute@rainer-klute.de
Körner Grund 24 Telefon: +49 172 2324824
D-44143 Dortmund Telefax: +49 231 5349423
OpenPGP fingerprint: E4E4386515EE0BED5C162FBB5343461584B5A42E
Re: Topic and Queue performances
Posted by Rob Davies <ra...@gmail.com>.
Messages are swapped to disk in version 4.2
cheers,
Rob
On 15 Jan 2007, at 21:39, Rainer Klute wrote:
> Rob Davies schrieb:
>> The big difference between topics and queues is that for queues, if
>> the messages have a persistent delivery mode, they are always
>> added to
>> the message store. This is only true for topics if a durable
>> subscriber exists and is interested in that topic.
>
> Which is not good. If many messages are added to the topic without
> being
> taken off from the broker by consumers, the memory is soaked up
> quickly,
> OutOfMemoryErrors occur, and the whole application gets corrupted. If
> the topic is too full, ActiveMQ should start to swap messages to
> external memory (disk). Sure, performance would be degraded but at
> least nothing will break.
>
> Best regards
> Rainer Klute
>
> Rainer Klute IT-Consulting GmbH
> Dipl.-Inform.
> Rainer Klute E-Mail: klute@rainer-klute.de
> Körner Grund 24 Telefon: +49 172 2324824
> D-44143 Dortmund Telefax: +49 231 5349423
>
> OpenPGP fingerprint: E4E4386515EE0BED5C162FBB5343461584B5A42E
>
>
Re: Topic and Queue performances
Posted by Rainer Klute <kl...@rainer-klute.de>.
Rob Davies schrieb:
> The big difference between topics and queues is that for queues, if
> the messages have a persistent delivery mode, they are always added to
> the message store. This is only true for topics if a durable
> subscriber exists and is interested in that topic.
Which is not good. If many messages are added to the topic without being
taken off from the broker by consumers, the memory is soaked up quickly,
OutOfMemoryErrors occur, and the whole application gets corrupted. If
the topic is too full, ActiveMQ should start to swap messages to
external memory (disk). Sure, performance would be degraded but at
least nothing will break.
Best regards
Rainer Klute
Rainer Klute IT-Consulting GmbH
Dipl.-Inform.
Rainer Klute E-Mail: klute@rainer-klute.de
Körner Grund 24 Telefon: +49 172 2324824
D-44143 Dortmund Telefax: +49 231 5349423
OpenPGP fingerprint: E4E4386515EE0BED5C162FBB5343461584B5A42E
Re: Topic and Queue performances
Posted by Rob Davies <ra...@gmail.com>.
The big difference between topics and queues is that for queues, if
the messages have a persistent delivery mode, they are always added
to the message store. This is only true for topics if a durable
subscriber exists and is interested in that topic.
By default, the message store used is the journal, which uses a jdbc
store for its long term storage.
cheers,
Rob
On 15 Jan 2007, at 19:39, James Strachan wrote:
> Are you positive you are using like for like - i.e. persistent
> messages for both queues and topics?
>
> You might also wanna try out using our maven based performance test
> harness which makes it easy to test out your environment - it might be
> worth seeing if your JMS code gets wildly different results which
> could indicate some issue in how you are using JMS...
>
> http://incubator.apache.org/activemq/activemq-performance-module-
> users-manual.html
>
>
> On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
>> Hello James,
>>
>> I understand very well that there are many factors that influence
>> such tests. In my
>> setup, I've tried to make the environment for both Queue and Topic
>> as similar as
>> possible, not to have the best results possible but to have
>> results that can be compared.
>>
>> As far as I know, the default config for Active MQ does not use
>> JDBC, or does it?
>> Also, on my tests, it's the Queue that is 40 times slower that the
>> Topic.
>>
>> My goal was more to have your opinion on the fact that, using the
>> same environment,
>> sending messages to a Queue is *much* slower that sending to a
>> Topic. Does that make
>> you thing that I've done something wrong, or is it something that
>> can be expected?
>>
>> I am currently checking out the source code of ActiveMQ to run
>> some tests as
>> described on your website[1] so I'll see how far my figures are
>> from your tests.
>>
>> Thanks for the answer,
>>
>> Sébastien
>>
>> [1] http://incubator.apache.org/activemq/jmeter-performance-
>> tests.html
>>
>>
>> James Strachan wrote:
>> > In general, it all depends
>> >
>> > http://incubator.apache.org/activemq/how-fast-is-activemq.html
>> >
>> > if using JDBC persistence, there is a bit more work required to
>> store
>> > a topic message than a queue (since topics need to maintain a
>> list of
>> > messages each consumer requires, whereas a queue is logically
>> > equivalent to a single row insert). Also obviously, if there are
>> more
>> > than one consumer, then the broker has more work to do with topics.
>> >
>> > But like many things, it all depends on what you are doing, how
>> you're
>> > doing it and what your environment is. YMMV
>> >
>> >
>> > On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
>> >> Hello,
>> >>
>> >> I checked on the list archive, but couldn't find the answer to my
>> >> current situation,
>> >> so here I am :)
>> >>
>> >> I've launched an ActiveMQ server on my machine, with the default
>> >> configuration stuff.
>> >> The only thing I've added is the jndi.properties file that I've
>> found
>> >> in the
>> >> documentation.
>> >>
>> >> This jndi config file helps create an example Queue and an example
>> >> Topic. I can
>> >> easily connect message producers and consumers to the Queue and
>> Topic.
>> >> The server, as
>> >> well as the two message producers and the two message
>> consumers, are
>> >> on my local machine.
>> >>
>> >> When I send messages to the Topic, I observe that the time it
>> takes to
>> >> send the
>> >> message is about 0.25 milliseconds. When I send messages to the
>> Queue,
>> >> the figure
>> >> goes up to 10 milliseconds. These figures are the average time
>> that is
>> >> observed when
>> >> sending 10'000 messages.
>> >> They are modified by the presence or absence of a consumer: it
>> takes
>> >> approximately
>> >> 10% less time when there is no consumer.
>> >>
>> >> Is it a normal behavior? I expected the performance between
>> Topic and
>> >> Queue to be
>> >> closer than that.
>> >>
>> >> Thanks for your help,
>> >>
>> >> Sébastien
>> >>
>> >> --
>> >> Sébastien Pennec
>> >> sebastien@qos.ch
>> >>
>> >> Logback: The reliable, generic, fast and flexible logging
>> framework
>> >> for Java.
>> >> http://logback.qos.ch/
>> >>
>> >
>> >
>>
>> --
>> Sébastien Pennec
>> sebastien@qos.ch
>>
>> Logback: The reliable, generic, fast and flexible logging
>> framework for Java.
>> http://logback.qos.ch/
>>
>
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
Re: Topic and Queue performances
Posted by James Strachan <ja...@gmail.com>.
Are you positive you are using like for like - i.e. persistent
messages for both queues and topics?
You might also wanna try out using our maven based performance test
harness which makes it easy to test out your environment - it might be
worth seeing if your JMS code gets wildly different results which
could indicate some issue in how you are using JMS...
http://incubator.apache.org/activemq/activemq-performance-module-users-manual.html
On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
> Hello James,
>
> I understand very well that there are many factors that influence such tests. In my
> setup, I've tried to make the environment for both Queue and Topic as similar as
> possible, not to have the best results possible but to have results that can be compared.
>
> As far as I know, the default config for Active MQ does not use JDBC, or does it?
> Also, on my tests, it's the Queue that is 40 times slower that the Topic.
>
> My goal was more to have your opinion on the fact that, using the same environment,
> sending messages to a Queue is *much* slower that sending to a Topic. Does that make
> you thing that I've done something wrong, or is it something that can be expected?
>
> I am currently checking out the source code of ActiveMQ to run some tests as
> described on your website[1] so I'll see how far my figures are from your tests.
>
> Thanks for the answer,
>
> Sébastien
>
> [1] http://incubator.apache.org/activemq/jmeter-performance-tests.html
>
>
> James Strachan wrote:
> > In general, it all depends
> >
> > http://incubator.apache.org/activemq/how-fast-is-activemq.html
> >
> > if using JDBC persistence, there is a bit more work required to store
> > a topic message than a queue (since topics need to maintain a list of
> > messages each consumer requires, whereas a queue is logically
> > equivalent to a single row insert). Also obviously, if there are more
> > than one consumer, then the broker has more work to do with topics.
> >
> > But like many things, it all depends on what you are doing, how you're
> > doing it and what your environment is. YMMV
> >
> >
> > On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
> >> Hello,
> >>
> >> I checked on the list archive, but couldn't find the answer to my
> >> current situation,
> >> so here I am :)
> >>
> >> I've launched an ActiveMQ server on my machine, with the default
> >> configuration stuff.
> >> The only thing I've added is the jndi.properties file that I've found
> >> in the
> >> documentation.
> >>
> >> This jndi config file helps create an example Queue and an example
> >> Topic. I can
> >> easily connect message producers and consumers to the Queue and Topic.
> >> The server, as
> >> well as the two message producers and the two message consumers, are
> >> on my local machine.
> >>
> >> When I send messages to the Topic, I observe that the time it takes to
> >> send the
> >> message is about 0.25 milliseconds. When I send messages to the Queue,
> >> the figure
> >> goes up to 10 milliseconds. These figures are the average time that is
> >> observed when
> >> sending 10'000 messages.
> >> They are modified by the presence or absence of a consumer: it takes
> >> approximately
> >> 10% less time when there is no consumer.
> >>
> >> Is it a normal behavior? I expected the performance between Topic and
> >> Queue to be
> >> closer than that.
> >>
> >> Thanks for your help,
> >>
> >> Sébastien
> >>
> >> --
> >> Sébastien Pennec
> >> sebastien@qos.ch
> >>
> >> Logback: The reliable, generic, fast and flexible logging framework
> >> for Java.
> >> http://logback.qos.ch/
> >>
> >
> >
>
> --
> Sébastien Pennec
> sebastien@qos.ch
>
> Logback: The reliable, generic, fast and flexible logging framework for Java.
> http://logback.qos.ch/
>
--
James
-------
http://radio.weblogs.com/0112098/
Re: Topic and Queue performances
Posted by Sebastien Pennec <se...@qos.ch>.
Hello James,
I understand very well that there are many factors that influence such tests. In my
setup, I've tried to make the environment for both Queue and Topic as similar as
possible, not to have the best results possible but to have results that can be compared.
As far as I know, the default config for Active MQ does not use JDBC, or does it?
Also, on my tests, it's the Queue that is 40 times slower that the Topic.
My goal was more to have your opinion on the fact that, using the same environment,
sending messages to a Queue is *much* slower that sending to a Topic. Does that make
you thing that I've done something wrong, or is it something that can be expected?
I am currently checking out the source code of ActiveMQ to run some tests as
described on your website[1] so I'll see how far my figures are from your tests.
Thanks for the answer,
Sébastien
[1] http://incubator.apache.org/activemq/jmeter-performance-tests.html
James Strachan wrote:
> In general, it all depends
>
> http://incubator.apache.org/activemq/how-fast-is-activemq.html
>
> if using JDBC persistence, there is a bit more work required to store
> a topic message than a queue (since topics need to maintain a list of
> messages each consumer requires, whereas a queue is logically
> equivalent to a single row insert). Also obviously, if there are more
> than one consumer, then the broker has more work to do with topics.
>
> But like many things, it all depends on what you are doing, how you're
> doing it and what your environment is. YMMV
>
>
> On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
>> Hello,
>>
>> I checked on the list archive, but couldn't find the answer to my
>> current situation,
>> so here I am :)
>>
>> I've launched an ActiveMQ server on my machine, with the default
>> configuration stuff.
>> The only thing I've added is the jndi.properties file that I've found
>> in the
>> documentation.
>>
>> This jndi config file helps create an example Queue and an example
>> Topic. I can
>> easily connect message producers and consumers to the Queue and Topic.
>> The server, as
>> well as the two message producers and the two message consumers, are
>> on my local machine.
>>
>> When I send messages to the Topic, I observe that the time it takes to
>> send the
>> message is about 0.25 milliseconds. When I send messages to the Queue,
>> the figure
>> goes up to 10 milliseconds. These figures are the average time that is
>> observed when
>> sending 10'000 messages.
>> They are modified by the presence or absence of a consumer: it takes
>> approximately
>> 10% less time when there is no consumer.
>>
>> Is it a normal behavior? I expected the performance between Topic and
>> Queue to be
>> closer than that.
>>
>> Thanks for your help,
>>
>> Sébastien
>>
>> --
>> Sébastien Pennec
>> sebastien@qos.ch
>>
>> Logback: The reliable, generic, fast and flexible logging framework
>> for Java.
>> http://logback.qos.ch/
>>
>
>
--
Sébastien Pennec
sebastien@qos.ch
Logback: The reliable, generic, fast and flexible logging framework for Java.
http://logback.qos.ch/
Re: Topic and Queue performances
Posted by James Strachan <ja...@gmail.com>.
In general, it all depends
http://incubator.apache.org/activemq/how-fast-is-activemq.html
if using JDBC persistence, there is a bit more work required to store
a topic message than a queue (since topics need to maintain a list of
messages each consumer requires, whereas a queue is logically
equivalent to a single row insert). Also obviously, if there are more
than one consumer, then the broker has more work to do with topics.
But like many things, it all depends on what you are doing, how you're
doing it and what your environment is. YMMV
On 1/15/07, Sebastien Pennec <se...@qos.ch> wrote:
> Hello,
>
> I checked on the list archive, but couldn't find the answer to my current situation,
> so here I am :)
>
> I've launched an ActiveMQ server on my machine, with the default configuration stuff.
> The only thing I've added is the jndi.properties file that I've found in the
> documentation.
>
> This jndi config file helps create an example Queue and an example Topic. I can
> easily connect message producers and consumers to the Queue and Topic. The server, as
> well as the two message producers and the two message consumers, are on my local machine.
>
> When I send messages to the Topic, I observe that the time it takes to send the
> message is about 0.25 milliseconds. When I send messages to the Queue, the figure
> goes up to 10 milliseconds. These figures are the average time that is observed when
> sending 10'000 messages.
> They are modified by the presence or absence of a consumer: it takes approximately
> 10% less time when there is no consumer.
>
> Is it a normal behavior? I expected the performance between Topic and Queue to be
> closer than that.
>
> Thanks for your help,
>
> Sébastien
>
> --
> Sébastien Pennec
> sebastien@qos.ch
>
> Logback: The reliable, generic, fast and flexible logging framework for Java.
> http://logback.qos.ch/
>
--
James
-------
http://radio.weblogs.com/0112098/