You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Clebert Suconic <cl...@gmail.com> on 2017/06/02 15:25:26 UTC
Re: why AvtiveMq is slowly than Kafka?
I have written this blog post here:
https://blogs.apache.org/activemq/entry/fast_messaging_with_artemis
You can combine that with using MAPPED at your journal type:
<!-- this could be ASYNCIO or NIO
-->
<journal-type>MAPPED</journal-type>
** BTW: I'm actually changing this text on "this could be ASYNCIO,
NIO.. adding MAPPED as well.
On Thu, Jun 1, 2017 at 9:57 PM, wangqinghuan <10...@qq.com> wrote:
> I don't understand why activemq is so slowly. I read some design document
> about KafKa. In Kafka,data is immediately written to a persistence log on
> the filesystem without flushing to disk. In effect this just means that it
> is transferred into the kernel's pagecache. Does activemq have this style of
> pagecache-centric design?
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
--
Clebert Suconic
Re: why AvtiveMq is slowly than Kafka?
Posted by Tim Bain <tb...@alumni.duke.edu>.
Were you comparing to ActiveMQ 5.x or to Artemis (or both)?
Another difference between Kafka and (I believe) both flavors of ActiveMQ
is that Kafka doesn't remove consumed messages, doesn't allow out-of-order
message acknowledgements (so if you want that, you have to implement it
yourself), and simply maintains consumer state as a number that is the
index of the message that consumer is currently on (so there's no need to
write an ack message to disk for each original message). Also, Kafka and
ActiveMQ Artemis support clusters of brokers for horizontal scalability,
whereas ActiveMQ 5.x doesn't. (But I've found that all aspects of
rebalancing partitions across additional brokers are harder, more manual,
and/or less fully-featured than they should be, so that ability of Kafka's
comes at a price.)
On Jun 2, 2017 10:51 AM, "nigro_franz" <ni...@gmail.com> wrote:
> Someone has summoned me :)
> I can add to set journal-datasync to false to make the Artemis journal
> behaviour to be the same of the default of Kafka (more here:
> https://activemq.apache.org/artemis/docs/2.1.0/persistence.html)
> Considering to have enough memory it enables the persistence to be
> effectively free :)
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4726951.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: why AvtiveMq is slowly than Kafka?
Posted by nigro_franz <ni...@gmail.com>.
Someone has summoned me :)
I can add to set journal-datasync to false to make the Artemis journal
behaviour to be the same of the default of Kafka (more here:
https://activemq.apache.org/artemis/docs/2.1.0/persistence.html)
Considering to have enough memory it enables the persistence to be
effectively free :)
--
View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4726951.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: why AvtiveMq is slowly than Kafka?
Posted by nigro_franz <ni...@gmail.com>.
Hi!
Just to understand...you've run a producer with no consumers?
--
View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4727139.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: why AvtiveMq is slowly than Kafka?
Posted by nigro_franz <ni...@gmail.com>.
wangqinghuan wrote
> why artemis maintains amounts of msgs in jvm space and needs to enable
> paging to page messages into and out jvm memory?
Because each have different purposes and information about the messages:
- the journal is a commit log -> it doesn't contain all the runtime
informations of the messages
- the (JVM) on heap space (eg QueueImpl) is where a message live until
expired or delivered -> it doesn't contain the sequence of operation on a
message
The paging allow, as you rightly pointed to move the message out of the JVM
space to an off heap storage to make the life easier for the user that wan't
a bound HEAP size and the most of GCs that are not very skilled (until new
ones will arrive) to deal with huge quantity of on heap references live for
long time.
--
View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4727166.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: why AvtiveMq is slowly than Kafka?
Posted by Clebert Suconic <cl...@gmail.com>.
I think you are confusing memory cache with paging on the message system.
Messages are kept in the memory for fast delivery unless you page then.
The message broker can be configured to start writing messages on disk for
later reading.
In a sense paging becomes like Kafka partition when paging kicks in. (I'm
not trying to make comparisons. Saying that just for educational purposes
please)
So if you plan no consumers. Configure paging on the address settings.
Look at he manual for paging.
On Wed, Jun 7, 2017 at 11:24 AM wangqinghuan <10...@qq.com> wrote:
> I understand that mmap against page cache , os helps swap page between
> memory and disk.So less memory will be used by jvm.
> why artemis maintains amounts of msgs in jvm space and needs to enable
> paging to page messages into and out jvm memory?
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4727164.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
Clebert Suconic
Re: why AvtiveMq is slowly than Kafka?
Posted by wangqinghuan <10...@qq.com>.
I understand that mmap against page cache , os helps swap page between
memory and disk.So less memory will be used by jvm.
why artemis maintains amounts of msgs in jvm space and needs to enable
paging to page messages into and out jvm memory?
--
View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4727164.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: why AvtiveMq is slowly than Kafka?
Posted by Clebert Suconic <cl...@gmail.com>.
You need to enable paging if you don't want a consumer.
Artemis is a messaging system. It's meant to be fast delivery with messages
passing.
If you want to store. No consumers. Enable paging.
On Wed, Jun 7, 2017 at 5:01 AM wangqinghuan <10...@qq.com> wrote:
> @clebertsuconic
> I run the demo in document "fast_messaging_with_artemis". Just started, the
> producer takes less 2s to send 100k messages.
> But as the number of messages stored in journal file increasing (more
> than
> 1 millions in journal file). Producer takes more than 30s to send 100k
> messages.
> This means , performance will occur a sharp decline when there are lots of
> msgs in journal file. Any suggestion for this scene? I overwrite the
> journal-type from NIO to ASYNCIO(or MAPPED),but no effect.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4727137.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
Clebert Suconic
Re: why AvtiveMq is slowly than Kafka?
Posted by wangqinghuan <10...@qq.com>.
@clebertsuconic
I run the demo in document "fast_messaging_with_artemis". Just started, the
producer takes less 2s to send 100k messages.
But as the number of messages stored in journal file increasing (more than
1 millions in journal file). Producer takes more than 30s to send 100k
messages.
This means , performance will occur a sharp decline when there are lots of
msgs in journal file. Any suggestion for this scene? I overwrite the
journal-type from NIO to ASYNCIO(or MAPPED),but no effect.
--
View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4727137.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: why AvtiveMq is slowly than Kafka?
Posted by Clebert Suconic <cl...@gmail.com>.
Also: Notice that with Kafka you always require a write on the disk...
if you are sending telemetry that you afford losing you may also opt
for non persistent.
On Fri, Jun 2, 2017 at 11:25 AM, Clebert Suconic
<cl...@gmail.com> wrote:
> I have written this blog post here:
>
> https://blogs.apache.org/activemq/entry/fast_messaging_with_artemis
>
>
> You can combine that with using MAPPED at your journal type:
>
>
>
>
> <!-- this could be ASYNCIO or NIO
>
> -->
>
> <journal-type>MAPPED</journal-type>
>
>
>
> ** BTW: I'm actually changing this text on "this could be ASYNCIO,
> NIO.. adding MAPPED as well.
>
> On Thu, Jun 1, 2017 at 9:57 PM, wangqinghuan <10...@qq.com> wrote:
>> I don't understand why activemq is so slowly. I read some design document
>> about KafKa. In Kafka,data is immediately written to a persistence log on
>> the filesystem without flushing to disk. In effect this just means that it
>> is transferred into the kernel's pagecache. Does activemq have this style of
>> pagecache-centric design?
>>
>>
>>
>>
>> --
>> View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
>
> --
> Clebert Suconic
--
Clebert Suconic
Re: why AvtiveMq is slowly than Kafka?
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
I just wanted to point out to Jakub's new book (which is free) that
talks about both ActiveMQ and Kafka and how they are different
http://www.jakubkorab.net/2017/06/new-book-understanding-message-brokers.html
On Mon, Jun 5, 2017 at 1:45 PM, Christopher Shannon
<ch...@gmail.com> wrote:
> You can't compare Kafka to a JMS type message broker. Kafka is completely
> different.
>
> Kafka is a system that scales horizontally and is essentially a big
> write-ahead log and breaks up the topics into partitions across many
> servers so they can be scanned concurrently. This allows insane message
> rates but the trade off is that the feature set is much less...there are no
> features like message acknowledgement (messages are not deleted, they are
> aged off and a client can seek to any point in the log), message
> expiration, scheduled messages, transactions (although transaction support
> is currently being worked on) etc which offloads a lot of work that a
> typical message broker has to do. Kafka clusters can scale to thousands
> of nodes and handle millions of messages per second.
>
> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
> at a rate of thousands per second.
>
> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <cl...@gmail.com>
> wrote:
>
>> For the use case you're after. (No hard syncs). Mmap is a good candidate.
>> Probably better.
>>
>>
>> Libaio was engineered the case where you hard sync with callbacks from the
>> Linux os
>> On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
>>
>> > hi clebertsuconic:
>> > i read the blog
>> > https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
>> > By default Apache ActiveMQ Artemis will try and use an AIO journal.But it
>> > seems like that Mmap is also a good implemention.which one gives more
>> > performance?
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> > http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-
>> tp4726911p4726992.html
>> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >
>> --
>> Clebert Suconic
>>
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: why AvtiveMq is slowly than Kafka?
Posted by andytaylor <an...@gmail.com>.
well said
On 5 June 2017 at 12:31, christopher.l.shannon [via ActiveMQ] <
ml+s2283324n4727011h21@n4.nabble.com> wrote:
> You can't compare Kafka to a JMS type message broker. Kafka is completely
> different.
>
> Kafka is a system that scales horizontally and is essentially a big
> write-ahead log and breaks up the topics into partitions across many
> servers so they can be scanned concurrently. This allows insane message
> rates but the trade off is that the feature set is much less...there are
> no
> features like message acknowledgement (messages are not deleted, they are
> aged off and a client can seek to any point in the log), message
> expiration, scheduled messages, transactions (although transaction support
> is currently being worked on) etc which offloads a lot of work that a
> typical message broker has to do. Kafka clusters can scale to thousands
> of nodes and handle millions of messages per second.
>
> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
> at a rate of thousands per second.
>
> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4727011&i=0>>
> wrote:
>
> > For the use case you're after. (No hard syncs). Mmap is a good
> candidate.
> > Probably better.
> >
> >
> > Libaio was engineered the case where you hard sync with callbacks from
> the
> > Linux os
> > On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=4727011&i=1>> wrote:
> >
> > > hi clebertsuconic:
> > > i read the blog
> > > https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
> > > By default Apache ActiveMQ Artemis will try and use an AIO journal.But
> it
> > > seems like that Mmap is also a good implemention.which one gives more
> > > performance?
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
> slowly-than-Kafka-
> > tp4726911p4726992.html
> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> > >
> > --
> > Clebert Suconic
> >
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-
> tp4726911p4727011.html
> To start a new topic under ActiveMQ - User, email
> ml+s2283324n2341805h35@n4.nabble.com
> To unsubscribe from ActiveMQ - User, click here
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2341805&code=YW5keS50YXlsczY3QGdtYWlsLmNvbXwyMzQxODA1fC05MDE1NDk1MzM=>
> .
> NAML
> <http://activemq.2283324.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4727048.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: why AvtiveMq is slowly than Kafka?
Posted by Justin Bertram <jb...@apache.org>.
I'm not aware of any SpecJMS results using any branch of Artemis. It'd be nice to have, but one basically has to set up a lab with dedicated hardware, set up the software, run it, etc. It's a fairly involved process from what I understand.
Justin
----- Original Message -----
From: "Michael André Pearce" <mi...@me.com>
To: users@activemq.apache.org
Sent: Monday, June 5, 2017 11:19:40 AM
Subject: Re: why AvtiveMq is slowly than Kafka?
@Justin
As you noted, with all the changes since hornetq became artemis, is there an updated run of that same JMSspec but on 2.x artemis?
It be great if there was. Just to see.
Sent from my iPhone
> On 5 Jun 2017, at 16:53, Justin Bertram <jb...@apache.org> wrote:
>
> Couple of things...
>
> 1) I agree 100% that comparing Kafka to message brokers like ActiveMQ is like comparing apples to oranges. They are really quite different.
> 2) My point wasn't necessarily to compare Artemis' performance with Kafka. My reply was simply in response to your assertion that, "Any standalone broker like ActiveMQ, Artemis, etc is going to be measured at a rate of thousands per second." I tried to make that clear by quoting this specific statement, but perhaps my intent wasn't as clear as I'd hoped.
> 3) I wasn't attempting to mislead anyone. I apologize if it came across that way.
> 4) The chance of hitting 8 million messages per second is greater than zero - at least as it's measured by SpecJMS. The benchmark results proves this. Of course, as with all things performance related the devil is in the details. Perhaps SpecJMS is measuring throughput in a different way than you might in your own test. In any event, "the most important requirement for the SPECjms2007 benchmark is that it is based on a representative workload scenario including a representative set of interactions, message types, message sizes and message delivery modes." [1]
> 5) Certainly fully synchronous and persistent use-cases will reduce performance, but many use-cases don't require that.
> 6) Clebert's "benchmark" (both code and operating environment) is quite different from the SpecJMS results I linked.
>
>
> Justin
>
> [1] https://www.spec.org/jms2007/docs/DesignDocument.html#S1
>
> ----- Original Message -----
> From: "Christopher Shannon" <ch...@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, June 5, 2017 8:28:50 AM
> Subject: Re: why AvtiveMq is slowly than Kafka?
>
> That may be what the benchmark says but there is zero chance of hitting 8
> million messages per second, especially persistent messages. Take a look
> at Clebert's blog:
> http://clebertsuconic.blogspot.com/2016/12/50k-persistent-messages-per-second-on.html
> And that required making the producer completely async.
>
> Even non-persistent messages, I think under most use cases and hardware I
> would be amazed if the performance was able to go past 100k messages a
> second.
>
> This is not to bash Artemis...it is very fast for a message broker but it
> is completely misleading to try and compare real world performance of a
> standalone broker (Artemis, ActiveMQ, EMS, etc) with something like Kafka.
>
>
>
> On Mon, Jun 5, 2017 at 9:03 AM, Justin Bertram <jb...@apache.org> wrote:
>
>>> Any standalone broker like ActiveMQ, Artemis, etc is going to be
>> measured at a rate of thousands per second.
>>
>> For what it's worth, HornetQ (upon which Artemis is based) achieved over 8
>> million messages per second on SpecJMS [1]. I would expect Artemis'
>> performance to be comparable (or better given some enhancements put in
>> place since then).
>>
>>
>> Justin
>>
>> [1] http://hornetq.blogspot.com/2011/07/82-million-messages-
>> second-with-specjms.html
>>
>> ----- Original Message -----
>> From: "Christopher Shannon" <ch...@gmail.com>
>> To: users@activemq.apache.org
>> Sent: Monday, June 5, 2017 6:45:26 AM
>> Subject: Re: why AvtiveMq is slowly than Kafka?
>>
>> You can't compare Kafka to a JMS type message broker. Kafka is completely
>> different.
>>
>> Kafka is a system that scales horizontally and is essentially a big
>> write-ahead log and breaks up the topics into partitions across many
>> servers so they can be scanned concurrently. This allows insane message
>> rates but the trade off is that the feature set is much less...there are no
>> features like message acknowledgement (messages are not deleted, they are
>> aged off and a client can seek to any point in the log), message
>> expiration, scheduled messages, transactions (although transaction support
>> is currently being worked on) etc which offloads a lot of work that a
>> typical message broker has to do. Kafka clusters can scale to thousands
>> of nodes and handle millions of messages per second.
>>
>> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
>> at a rate of thousands per second.
>>
>> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <clebert.suconic@gmail.com
>>>
>> wrote:
>>
>>> For the use case you're after. (No hard syncs). Mmap is a good candidate.
>>> Probably better.
>>>
>>>
>>> Libaio was engineered the case where you hard sync with callbacks from
>> the
>>> Linux os
>>>> On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
>>>>
>>>> hi clebertsuconic:
>>>> i read the blog
>>>> https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
>>>> By default Apache ActiveMQ Artemis will try and use an AIO journal.But
>> it
>>>> seems like that Mmap is also a good implemention.which one gives more
>>>> performance?
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
>> slowly-than-Kafka-
>>> tp4726911p4726992.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>> --
>>> Clebert Suconic
>>>
>>
Re: why AvtiveMq is slowly than Kafka?
Posted by Michael André Pearce <mi...@me.com>.
I totally agree. Just be good to somehow quantify and highlight that hard work and how much the project has improved from what was compared to today. And I guess next year again.
I guess if running that spec is too expensive re resource it's not worth it.
Sent from my iPhone
> On 5 Jun 2017, at 22:27, Clebert Suconic <cl...@gmail.com> wrote:
>
> @Michael Pearce: The native module on Artemis is totally rewritten...
> hornetq libaio used a lot of malloc. Artemis is only calling malloc
> once and reusing them on a pool... a lot less locking happening...
> just the libaio is better..
>
>
> For instance.. an issue that you had with 512 bytes blocks recently..
> it wouldn't even be possible to be fixed with hornetq.
>
>
> I don't have number with specj.. but I remember one microbenchmark I
> had got 30% better with Artemis after the native update.. just by
> itself...
>
>
> ActiveMQ Artemis is getting further and further away from hornetq.
> it's evolving nicely IMO.
>
>
> On Mon, Jun 5, 2017 at 12:19 PM, Michael André Pearce
> <mi...@me.com> wrote:
>> @Justin
>>
>> As you noted, with all the changes since hornetq became artemis, is there an updated run of that same JMSspec but on 2.x artemis?
>>
>> It be great if there was. Just to see.
>>
>> Sent from my iPhone
>>
>>> On 5 Jun 2017, at 16:53, Justin Bertram <jb...@apache.org> wrote:
>>>
>>> Couple of things...
>>>
>>> 1) I agree 100% that comparing Kafka to message brokers like ActiveMQ is like comparing apples to oranges. They are really quite different.
>>> 2) My point wasn't necessarily to compare Artemis' performance with Kafka. My reply was simply in response to your assertion that, "Any standalone broker like ActiveMQ, Artemis, etc is going to be measured at a rate of thousands per second." I tried to make that clear by quoting this specific statement, but perhaps my intent wasn't as clear as I'd hoped.
>>> 3) I wasn't attempting to mislead anyone. I apologize if it came across that way.
>>> 4) The chance of hitting 8 million messages per second is greater than zero - at least as it's measured by SpecJMS. The benchmark results proves this. Of course, as with all things performance related the devil is in the details. Perhaps SpecJMS is measuring throughput in a different way than you might in your own test. In any event, "the most important requirement for the SPECjms2007 benchmark is that it is based on a representative workload scenario including a representative set of interactions, message types, message sizes and message delivery modes." [1]
>>> 5) Certainly fully synchronous and persistent use-cases will reduce performance, but many use-cases don't require that.
>>> 6) Clebert's "benchmark" (both code and operating environment) is quite different from the SpecJMS results I linked.
>>>
>>>
>>> Justin
>>>
>>> [1] https://www.spec.org/jms2007/docs/DesignDocument.html#S1
>>>
>>> ----- Original Message -----
>>> From: "Christopher Shannon" <ch...@gmail.com>
>>> To: users@activemq.apache.org
>>> Sent: Monday, June 5, 2017 8:28:50 AM
>>> Subject: Re: why AvtiveMq is slowly than Kafka?
>>>
>>> That may be what the benchmark says but there is zero chance of hitting 8
>>> million messages per second, especially persistent messages. Take a look
>>> at Clebert's blog:
>>> http://clebertsuconic.blogspot.com/2016/12/50k-persistent-messages-per-second-on.html
>>> And that required making the producer completely async.
>>>
>>> Even non-persistent messages, I think under most use cases and hardware I
>>> would be amazed if the performance was able to go past 100k messages a
>>> second.
>>>
>>> This is not to bash Artemis...it is very fast for a message broker but it
>>> is completely misleading to try and compare real world performance of a
>>> standalone broker (Artemis, ActiveMQ, EMS, etc) with something like Kafka.
>>>
>>>
>>>
>>> On Mon, Jun 5, 2017 at 9:03 AM, Justin Bertram <jb...@apache.org> wrote:
>>>
>>>>> Any standalone broker like ActiveMQ, Artemis, etc is going to be
>>>> measured at a rate of thousands per second.
>>>>
>>>> For what it's worth, HornetQ (upon which Artemis is based) achieved over 8
>>>> million messages per second on SpecJMS [1]. I would expect Artemis'
>>>> performance to be comparable (or better given some enhancements put in
>>>> place since then).
>>>>
>>>>
>>>> Justin
>>>>
>>>> [1] http://hornetq.blogspot.com/2011/07/82-million-messages-
>>>> second-with-specjms.html
>>>>
>>>> ----- Original Message -----
>>>> From: "Christopher Shannon" <ch...@gmail.com>
>>>> To: users@activemq.apache.org
>>>> Sent: Monday, June 5, 2017 6:45:26 AM
>>>> Subject: Re: why AvtiveMq is slowly than Kafka?
>>>>
>>>> You can't compare Kafka to a JMS type message broker. Kafka is completely
>>>> different.
>>>>
>>>> Kafka is a system that scales horizontally and is essentially a big
>>>> write-ahead log and breaks up the topics into partitions across many
>>>> servers so they can be scanned concurrently. This allows insane message
>>>> rates but the trade off is that the feature set is much less...there are no
>>>> features like message acknowledgement (messages are not deleted, they are
>>>> aged off and a client can seek to any point in the log), message
>>>> expiration, scheduled messages, transactions (although transaction support
>>>> is currently being worked on) etc which offloads a lot of work that a
>>>> typical message broker has to do. Kafka clusters can scale to thousands
>>>> of nodes and handle millions of messages per second.
>>>>
>>>> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
>>>> at a rate of thousands per second.
>>>>
>>>> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <clebert.suconic@gmail.com
>>>>>
>>>> wrote:
>>>>
>>>>> For the use case you're after. (No hard syncs). Mmap is a good candidate.
>>>>> Probably better.
>>>>>
>>>>>
>>>>> Libaio was engineered the case where you hard sync with callbacks from
>>>> the
>>>>> Linux os
>>>>>> On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
>>>>>>
>>>>>> hi clebertsuconic:
>>>>>> i read the blog
>>>>>> https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
>>>>>> By default Apache ActiveMQ Artemis will try and use an AIO journal.But
>>>> it
>>>>>> seems like that Mmap is also a good implemention.which one gives more
>>>>>> performance?
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
>>>> slowly-than-Kafka-
>>>>> tp4726911p4726992.html
>>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>>
>>>>> --
>>>>> Clebert Suconic
>>>>>
>>>>
>
>
>
> --
> Clebert Suconic
Re: why AvtiveMq is slowly than Kafka?
Posted by Clebert Suconic <cl...@gmail.com>.
@Michael Pearce: The native module on Artemis is totally rewritten...
hornetq libaio used a lot of malloc. Artemis is only calling malloc
once and reusing them on a pool... a lot less locking happening...
just the libaio is better..
For instance.. an issue that you had with 512 bytes blocks recently..
it wouldn't even be possible to be fixed with hornetq.
I don't have number with specj.. but I remember one microbenchmark I
had got 30% better with Artemis after the native update.. just by
itself...
ActiveMQ Artemis is getting further and further away from hornetq.
it's evolving nicely IMO.
On Mon, Jun 5, 2017 at 12:19 PM, Michael André Pearce
<mi...@me.com> wrote:
> @Justin
>
> As you noted, with all the changes since hornetq became artemis, is there an updated run of that same JMSspec but on 2.x artemis?
>
> It be great if there was. Just to see.
>
> Sent from my iPhone
>
>> On 5 Jun 2017, at 16:53, Justin Bertram <jb...@apache.org> wrote:
>>
>> Couple of things...
>>
>> 1) I agree 100% that comparing Kafka to message brokers like ActiveMQ is like comparing apples to oranges. They are really quite different.
>> 2) My point wasn't necessarily to compare Artemis' performance with Kafka. My reply was simply in response to your assertion that, "Any standalone broker like ActiveMQ, Artemis, etc is going to be measured at a rate of thousands per second." I tried to make that clear by quoting this specific statement, but perhaps my intent wasn't as clear as I'd hoped.
>> 3) I wasn't attempting to mislead anyone. I apologize if it came across that way.
>> 4) The chance of hitting 8 million messages per second is greater than zero - at least as it's measured by SpecJMS. The benchmark results proves this. Of course, as with all things performance related the devil is in the details. Perhaps SpecJMS is measuring throughput in a different way than you might in your own test. In any event, "the most important requirement for the SPECjms2007 benchmark is that it is based on a representative workload scenario including a representative set of interactions, message types, message sizes and message delivery modes." [1]
>> 5) Certainly fully synchronous and persistent use-cases will reduce performance, but many use-cases don't require that.
>> 6) Clebert's "benchmark" (both code and operating environment) is quite different from the SpecJMS results I linked.
>>
>>
>> Justin
>>
>> [1] https://www.spec.org/jms2007/docs/DesignDocument.html#S1
>>
>> ----- Original Message -----
>> From: "Christopher Shannon" <ch...@gmail.com>
>> To: users@activemq.apache.org
>> Sent: Monday, June 5, 2017 8:28:50 AM
>> Subject: Re: why AvtiveMq is slowly than Kafka?
>>
>> That may be what the benchmark says but there is zero chance of hitting 8
>> million messages per second, especially persistent messages. Take a look
>> at Clebert's blog:
>> http://clebertsuconic.blogspot.com/2016/12/50k-persistent-messages-per-second-on.html
>> And that required making the producer completely async.
>>
>> Even non-persistent messages, I think under most use cases and hardware I
>> would be amazed if the performance was able to go past 100k messages a
>> second.
>>
>> This is not to bash Artemis...it is very fast for a message broker but it
>> is completely misleading to try and compare real world performance of a
>> standalone broker (Artemis, ActiveMQ, EMS, etc) with something like Kafka.
>>
>>
>>
>> On Mon, Jun 5, 2017 at 9:03 AM, Justin Bertram <jb...@apache.org> wrote:
>>
>>>> Any standalone broker like ActiveMQ, Artemis, etc is going to be
>>> measured at a rate of thousands per second.
>>>
>>> For what it's worth, HornetQ (upon which Artemis is based) achieved over 8
>>> million messages per second on SpecJMS [1]. I would expect Artemis'
>>> performance to be comparable (or better given some enhancements put in
>>> place since then).
>>>
>>>
>>> Justin
>>>
>>> [1] http://hornetq.blogspot.com/2011/07/82-million-messages-
>>> second-with-specjms.html
>>>
>>> ----- Original Message -----
>>> From: "Christopher Shannon" <ch...@gmail.com>
>>> To: users@activemq.apache.org
>>> Sent: Monday, June 5, 2017 6:45:26 AM
>>> Subject: Re: why AvtiveMq is slowly than Kafka?
>>>
>>> You can't compare Kafka to a JMS type message broker. Kafka is completely
>>> different.
>>>
>>> Kafka is a system that scales horizontally and is essentially a big
>>> write-ahead log and breaks up the topics into partitions across many
>>> servers so they can be scanned concurrently. This allows insane message
>>> rates but the trade off is that the feature set is much less...there are no
>>> features like message acknowledgement (messages are not deleted, they are
>>> aged off and a client can seek to any point in the log), message
>>> expiration, scheduled messages, transactions (although transaction support
>>> is currently being worked on) etc which offloads a lot of work that a
>>> typical message broker has to do. Kafka clusters can scale to thousands
>>> of nodes and handle millions of messages per second.
>>>
>>> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
>>> at a rate of thousands per second.
>>>
>>> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <clebert.suconic@gmail.com
>>>>
>>> wrote:
>>>
>>>> For the use case you're after. (No hard syncs). Mmap is a good candidate.
>>>> Probably better.
>>>>
>>>>
>>>> Libaio was engineered the case where you hard sync with callbacks from
>>> the
>>>> Linux os
>>>>> On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
>>>>>
>>>>> hi clebertsuconic:
>>>>> i read the blog
>>>>> https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
>>>>> By default Apache ActiveMQ Artemis will try and use an AIO journal.But
>>> it
>>>>> seems like that Mmap is also a good implemention.which one gives more
>>>>> performance?
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
>>> slowly-than-Kafka-
>>>> tp4726911p4726992.html
>>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>>
>>>> --
>>>> Clebert Suconic
>>>>
>>>
--
Clebert Suconic
Re: why AvtiveMq is slowly than Kafka?
Posted by Michael André Pearce <mi...@me.com>.
@Justin
As you noted, with all the changes since hornetq became artemis, is there an updated run of that same JMSspec but on 2.x artemis?
It be great if there was. Just to see.
Sent from my iPhone
> On 5 Jun 2017, at 16:53, Justin Bertram <jb...@apache.org> wrote:
>
> Couple of things...
>
> 1) I agree 100% that comparing Kafka to message brokers like ActiveMQ is like comparing apples to oranges. They are really quite different.
> 2) My point wasn't necessarily to compare Artemis' performance with Kafka. My reply was simply in response to your assertion that, "Any standalone broker like ActiveMQ, Artemis, etc is going to be measured at a rate of thousands per second." I tried to make that clear by quoting this specific statement, but perhaps my intent wasn't as clear as I'd hoped.
> 3) I wasn't attempting to mislead anyone. I apologize if it came across that way.
> 4) The chance of hitting 8 million messages per second is greater than zero - at least as it's measured by SpecJMS. The benchmark results proves this. Of course, as with all things performance related the devil is in the details. Perhaps SpecJMS is measuring throughput in a different way than you might in your own test. In any event, "the most important requirement for the SPECjms2007 benchmark is that it is based on a representative workload scenario including a representative set of interactions, message types, message sizes and message delivery modes." [1]
> 5) Certainly fully synchronous and persistent use-cases will reduce performance, but many use-cases don't require that.
> 6) Clebert's "benchmark" (both code and operating environment) is quite different from the SpecJMS results I linked.
>
>
> Justin
>
> [1] https://www.spec.org/jms2007/docs/DesignDocument.html#S1
>
> ----- Original Message -----
> From: "Christopher Shannon" <ch...@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, June 5, 2017 8:28:50 AM
> Subject: Re: why AvtiveMq is slowly than Kafka?
>
> That may be what the benchmark says but there is zero chance of hitting 8
> million messages per second, especially persistent messages. Take a look
> at Clebert's blog:
> http://clebertsuconic.blogspot.com/2016/12/50k-persistent-messages-per-second-on.html
> And that required making the producer completely async.
>
> Even non-persistent messages, I think under most use cases and hardware I
> would be amazed if the performance was able to go past 100k messages a
> second.
>
> This is not to bash Artemis...it is very fast for a message broker but it
> is completely misleading to try and compare real world performance of a
> standalone broker (Artemis, ActiveMQ, EMS, etc) with something like Kafka.
>
>
>
> On Mon, Jun 5, 2017 at 9:03 AM, Justin Bertram <jb...@apache.org> wrote:
>
>>> Any standalone broker like ActiveMQ, Artemis, etc is going to be
>> measured at a rate of thousands per second.
>>
>> For what it's worth, HornetQ (upon which Artemis is based) achieved over 8
>> million messages per second on SpecJMS [1]. I would expect Artemis'
>> performance to be comparable (or better given some enhancements put in
>> place since then).
>>
>>
>> Justin
>>
>> [1] http://hornetq.blogspot.com/2011/07/82-million-messages-
>> second-with-specjms.html
>>
>> ----- Original Message -----
>> From: "Christopher Shannon" <ch...@gmail.com>
>> To: users@activemq.apache.org
>> Sent: Monday, June 5, 2017 6:45:26 AM
>> Subject: Re: why AvtiveMq is slowly than Kafka?
>>
>> You can't compare Kafka to a JMS type message broker. Kafka is completely
>> different.
>>
>> Kafka is a system that scales horizontally and is essentially a big
>> write-ahead log and breaks up the topics into partitions across many
>> servers so they can be scanned concurrently. This allows insane message
>> rates but the trade off is that the feature set is much less...there are no
>> features like message acknowledgement (messages are not deleted, they are
>> aged off and a client can seek to any point in the log), message
>> expiration, scheduled messages, transactions (although transaction support
>> is currently being worked on) etc which offloads a lot of work that a
>> typical message broker has to do. Kafka clusters can scale to thousands
>> of nodes and handle millions of messages per second.
>>
>> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
>> at a rate of thousands per second.
>>
>> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <clebert.suconic@gmail.com
>>>
>> wrote:
>>
>>> For the use case you're after. (No hard syncs). Mmap is a good candidate.
>>> Probably better.
>>>
>>>
>>> Libaio was engineered the case where you hard sync with callbacks from
>> the
>>> Linux os
>>>> On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
>>>>
>>>> hi clebertsuconic:
>>>> i read the blog
>>>> https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
>>>> By default Apache ActiveMQ Artemis will try and use an AIO journal.But
>> it
>>>> seems like that Mmap is also a good implemention.which one gives more
>>>> performance?
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
>> slowly-than-Kafka-
>>> tp4726911p4726992.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>>
>>> --
>>> Clebert Suconic
>>>
>>
Re: why AvtiveMq is slowly than Kafka?
Posted by Christopher Shannon <ch...@gmail.com>.
Justin,
Thanks for linking the design document for the benchmark test I will need
to take a look and see how what the test is for the results to come up with
that kind of speed.
Based on my own experience using several types of brokers a typical message
rate under any scenario (non-persistent and async producers included) is
going to be much slower and come no where near that number.
On Mon, Jun 5, 2017 at 11:53 AM, Justin Bertram <jb...@apache.org> wrote:
> Couple of things...
>
> 1) I agree 100% that comparing Kafka to message brokers like ActiveMQ is
> like comparing apples to oranges. They are really quite different.
> 2) My point wasn't necessarily to compare Artemis' performance with
> Kafka. My reply was simply in response to your assertion that, "Any
> standalone broker like ActiveMQ, Artemis, etc is going to be measured at a
> rate of thousands per second." I tried to make that clear by quoting this
> specific statement, but perhaps my intent wasn't as clear as I'd hoped.
> 3) I wasn't attempting to mislead anyone. I apologize if it came across
> that way.
> 4) The chance of hitting 8 million messages per second is greater than
> zero - at least as it's measured by SpecJMS. The benchmark results proves
> this. Of course, as with all things performance related the devil is in
> the details. Perhaps SpecJMS is measuring throughput in a different way
> than you might in your own test. In any event, "the most important
> requirement for the SPECjms2007 benchmark is that it is based on a
> representative workload scenario including a representative set of
> interactions, message types, message sizes and message delivery modes." [1]
> 5) Certainly fully synchronous and persistent use-cases will reduce
> performance, but many use-cases don't require that.
> 6) Clebert's "benchmark" (both code and operating environment) is quite
> different from the SpecJMS results I linked.
>
>
> Justin
>
> [1] https://www.spec.org/jms2007/docs/DesignDocument.html#S1
>
> ----- Original Message -----
> From: "Christopher Shannon" <ch...@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, June 5, 2017 8:28:50 AM
> Subject: Re: why AvtiveMq is slowly than Kafka?
>
> That may be what the benchmark says but there is zero chance of hitting 8
> million messages per second, especially persistent messages. Take a look
> at Clebert's blog:
> http://clebertsuconic.blogspot.com/2016/12/50k-persistent-messages-per-
> second-on.html
> And that required making the producer completely async.
>
> Even non-persistent messages, I think under most use cases and hardware I
> would be amazed if the performance was able to go past 100k messages a
> second.
>
> This is not to bash Artemis...it is very fast for a message broker but it
> is completely misleading to try and compare real world performance of a
> standalone broker (Artemis, ActiveMQ, EMS, etc) with something like Kafka.
>
>
>
> On Mon, Jun 5, 2017 at 9:03 AM, Justin Bertram <jb...@apache.org>
> wrote:
>
> > > Any standalone broker like ActiveMQ, Artemis, etc is going to be
> > measured at a rate of thousands per second.
> >
> > For what it's worth, HornetQ (upon which Artemis is based) achieved over
> 8
> > million messages per second on SpecJMS [1]. I would expect Artemis'
> > performance to be comparable (or better given some enhancements put in
> > place since then).
> >
> >
> > Justin
> >
> > [1] http://hornetq.blogspot.com/2011/07/82-million-messages-
> > second-with-specjms.html
> >
> > ----- Original Message -----
> > From: "Christopher Shannon" <ch...@gmail.com>
> > To: users@activemq.apache.org
> > Sent: Monday, June 5, 2017 6:45:26 AM
> > Subject: Re: why AvtiveMq is slowly than Kafka?
> >
> > You can't compare Kafka to a JMS type message broker. Kafka is
> completely
> > different.
> >
> > Kafka is a system that scales horizontally and is essentially a big
> > write-ahead log and breaks up the topics into partitions across many
> > servers so they can be scanned concurrently. This allows insane message
> > rates but the trade off is that the feature set is much less...there are
> no
> > features like message acknowledgement (messages are not deleted, they are
> > aged off and a client can seek to any point in the log), message
> > expiration, scheduled messages, transactions (although transaction
> support
> > is currently being worked on) etc which offloads a lot of work that a
> > typical message broker has to do. Kafka clusters can scale to thousands
> > of nodes and handle millions of messages per second.
> >
> > Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
> > at a rate of thousands per second.
> >
> > On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <
> clebert.suconic@gmail.com
> > >
> > wrote:
> >
> > > For the use case you're after. (No hard syncs). Mmap is a good
> candidate.
> > > Probably better.
> > >
> > >
> > > Libaio was engineered the case where you hard sync with callbacks from
> > the
> > > Linux os
> > > On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com>
> wrote:
> > >
> > > > hi clebertsuconic:
> > > > i read the blog
> > > > https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
> > > > By default Apache ActiveMQ Artemis will try and use an AIO
> journal.But
> > it
> > > > seems like that Mmap is also a good implemention.which one gives more
> > > > performance?
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > > http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
> > slowly-than-Kafka-
> > > tp4726911p4726992.html
> > > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> > > >
> > > --
> > > Clebert Suconic
> > >
> >
>
Re: why AvtiveMq is slowly than Kafka?
Posted by Justin Bertram <jb...@apache.org>.
Couple of things...
1) I agree 100% that comparing Kafka to message brokers like ActiveMQ is like comparing apples to oranges. They are really quite different.
2) My point wasn't necessarily to compare Artemis' performance with Kafka. My reply was simply in response to your assertion that, "Any standalone broker like ActiveMQ, Artemis, etc is going to be measured at a rate of thousands per second." I tried to make that clear by quoting this specific statement, but perhaps my intent wasn't as clear as I'd hoped.
3) I wasn't attempting to mislead anyone. I apologize if it came across that way.
4) The chance of hitting 8 million messages per second is greater than zero - at least as it's measured by SpecJMS. The benchmark results proves this. Of course, as with all things performance related the devil is in the details. Perhaps SpecJMS is measuring throughput in a different way than you might in your own test. In any event, "the most important requirement for the SPECjms2007 benchmark is that it is based on a representative workload scenario including a representative set of interactions, message types, message sizes and message delivery modes." [1]
5) Certainly fully synchronous and persistent use-cases will reduce performance, but many use-cases don't require that.
6) Clebert's "benchmark" (both code and operating environment) is quite different from the SpecJMS results I linked.
Justin
[1] https://www.spec.org/jms2007/docs/DesignDocument.html#S1
----- Original Message -----
From: "Christopher Shannon" <ch...@gmail.com>
To: users@activemq.apache.org
Sent: Monday, June 5, 2017 8:28:50 AM
Subject: Re: why AvtiveMq is slowly than Kafka?
That may be what the benchmark says but there is zero chance of hitting 8
million messages per second, especially persistent messages. Take a look
at Clebert's blog:
http://clebertsuconic.blogspot.com/2016/12/50k-persistent-messages-per-second-on.html
And that required making the producer completely async.
Even non-persistent messages, I think under most use cases and hardware I
would be amazed if the performance was able to go past 100k messages a
second.
This is not to bash Artemis...it is very fast for a message broker but it
is completely misleading to try and compare real world performance of a
standalone broker (Artemis, ActiveMQ, EMS, etc) with something like Kafka.
On Mon, Jun 5, 2017 at 9:03 AM, Justin Bertram <jb...@apache.org> wrote:
> > Any standalone broker like ActiveMQ, Artemis, etc is going to be
> measured at a rate of thousands per second.
>
> For what it's worth, HornetQ (upon which Artemis is based) achieved over 8
> million messages per second on SpecJMS [1]. I would expect Artemis'
> performance to be comparable (or better given some enhancements put in
> place since then).
>
>
> Justin
>
> [1] http://hornetq.blogspot.com/2011/07/82-million-messages-
> second-with-specjms.html
>
> ----- Original Message -----
> From: "Christopher Shannon" <ch...@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, June 5, 2017 6:45:26 AM
> Subject: Re: why AvtiveMq is slowly than Kafka?
>
> You can't compare Kafka to a JMS type message broker. Kafka is completely
> different.
>
> Kafka is a system that scales horizontally and is essentially a big
> write-ahead log and breaks up the topics into partitions across many
> servers so they can be scanned concurrently. This allows insane message
> rates but the trade off is that the feature set is much less...there are no
> features like message acknowledgement (messages are not deleted, they are
> aged off and a client can seek to any point in the log), message
> expiration, scheduled messages, transactions (although transaction support
> is currently being worked on) etc which offloads a lot of work that a
> typical message broker has to do. Kafka clusters can scale to thousands
> of nodes and handle millions of messages per second.
>
> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
> at a rate of thousands per second.
>
> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <clebert.suconic@gmail.com
> >
> wrote:
>
> > For the use case you're after. (No hard syncs). Mmap is a good candidate.
> > Probably better.
> >
> >
> > Libaio was engineered the case where you hard sync with callbacks from
> the
> > Linux os
> > On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
> >
> > > hi clebertsuconic:
> > > i read the blog
> > > https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
> > > By default Apache ActiveMQ Artemis will try and use an AIO journal.But
> it
> > > seems like that Mmap is also a good implemention.which one gives more
> > > performance?
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
> slowly-than-Kafka-
> > tp4726911p4726992.html
> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> > >
> > --
> > Clebert Suconic
> >
>
Re: why AvtiveMq is slowly than Kafka?
Posted by Christopher Shannon <ch...@gmail.com>.
That may be what the benchmark says but there is zero chance of hitting 8
million messages per second, especially persistent messages. Take a look
at Clebert's blog:
http://clebertsuconic.blogspot.com/2016/12/50k-persistent-messages-per-second-on.html
And that required making the producer completely async.
Even non-persistent messages, I think under most use cases and hardware I
would be amazed if the performance was able to go past 100k messages a
second.
This is not to bash Artemis...it is very fast for a message broker but it
is completely misleading to try and compare real world performance of a
standalone broker (Artemis, ActiveMQ, EMS, etc) with something like Kafka.
On Mon, Jun 5, 2017 at 9:03 AM, Justin Bertram <jb...@apache.org> wrote:
> > Any standalone broker like ActiveMQ, Artemis, etc is going to be
> measured at a rate of thousands per second.
>
> For what it's worth, HornetQ (upon which Artemis is based) achieved over 8
> million messages per second on SpecJMS [1]. I would expect Artemis'
> performance to be comparable (or better given some enhancements put in
> place since then).
>
>
> Justin
>
> [1] http://hornetq.blogspot.com/2011/07/82-million-messages-
> second-with-specjms.html
>
> ----- Original Message -----
> From: "Christopher Shannon" <ch...@gmail.com>
> To: users@activemq.apache.org
> Sent: Monday, June 5, 2017 6:45:26 AM
> Subject: Re: why AvtiveMq is slowly than Kafka?
>
> You can't compare Kafka to a JMS type message broker. Kafka is completely
> different.
>
> Kafka is a system that scales horizontally and is essentially a big
> write-ahead log and breaks up the topics into partitions across many
> servers so they can be scanned concurrently. This allows insane message
> rates but the trade off is that the feature set is much less...there are no
> features like message acknowledgement (messages are not deleted, they are
> aged off and a client can seek to any point in the log), message
> expiration, scheduled messages, transactions (although transaction support
> is currently being worked on) etc which offloads a lot of work that a
> typical message broker has to do. Kafka clusters can scale to thousands
> of nodes and handle millions of messages per second.
>
> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
> at a rate of thousands per second.
>
> On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <clebert.suconic@gmail.com
> >
> wrote:
>
> > For the use case you're after. (No hard syncs). Mmap is a good candidate.
> > Probably better.
> >
> >
> > Libaio was engineered the case where you hard sync with callbacks from
> the
> > Linux os
> > On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
> >
> > > hi clebertsuconic:
> > > i read the blog
> > > https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
> > > By default Apache ActiveMQ Artemis will try and use an AIO journal.But
> it
> > > seems like that Mmap is also a good implemention.which one gives more
> > > performance?
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-
> slowly-than-Kafka-
> > tp4726911p4726992.html
> > > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> > >
> > --
> > Clebert Suconic
> >
>
Re: why AvtiveMq is slowly than Kafka?
Posted by Justin Bertram <jb...@apache.org>.
> Any standalone broker like ActiveMQ, Artemis, etc is going to be measured at a rate of thousands per second.
For what it's worth, HornetQ (upon which Artemis is based) achieved over 8 million messages per second on SpecJMS [1]. I would expect Artemis' performance to be comparable (or better given some enhancements put in place since then).
Justin
[1] http://hornetq.blogspot.com/2011/07/82-million-messages-second-with-specjms.html
----- Original Message -----
From: "Christopher Shannon" <ch...@gmail.com>
To: users@activemq.apache.org
Sent: Monday, June 5, 2017 6:45:26 AM
Subject: Re: why AvtiveMq is slowly than Kafka?
You can't compare Kafka to a JMS type message broker. Kafka is completely
different.
Kafka is a system that scales horizontally and is essentially a big
write-ahead log and breaks up the topics into partitions across many
servers so they can be scanned concurrently. This allows insane message
rates but the trade off is that the feature set is much less...there are no
features like message acknowledgement (messages are not deleted, they are
aged off and a client can seek to any point in the log), message
expiration, scheduled messages, transactions (although transaction support
is currently being worked on) etc which offloads a lot of work that a
typical message broker has to do. Kafka clusters can scale to thousands
of nodes and handle millions of messages per second.
Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
at a rate of thousands per second.
On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <cl...@gmail.com>
wrote:
> For the use case you're after. (No hard syncs). Mmap is a good candidate.
> Probably better.
>
>
> Libaio was engineered the case where you hard sync with callbacks from the
> Linux os
> On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
>
> > hi clebertsuconic:
> > i read the blog
> > https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
> > By default Apache ActiveMQ Artemis will try and use an AIO journal.But it
> > seems like that Mmap is also a good implemention.which one gives more
> > performance?
> >
> >
> >
> > --
> > View this message in context:
> > http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-
> tp4726911p4726992.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
> --
> Clebert Suconic
>
Re: why AvtiveMq is slowly than Kafka?
Posted by Christopher Shannon <ch...@gmail.com>.
You can't compare Kafka to a JMS type message broker. Kafka is completely
different.
Kafka is a system that scales horizontally and is essentially a big
write-ahead log and breaks up the topics into partitions across many
servers so they can be scanned concurrently. This allows insane message
rates but the trade off is that the feature set is much less...there are no
features like message acknowledgement (messages are not deleted, they are
aged off and a client can seek to any point in the log), message
expiration, scheduled messages, transactions (although transaction support
is currently being worked on) etc which offloads a lot of work that a
typical message broker has to do. Kafka clusters can scale to thousands
of nodes and handle millions of messages per second.
Any standalone broker like ActiveMQ, Artemis, etc is going to be measured
at a rate of thousands per second.
On Sat, Jun 3, 2017 at 3:13 PM, Clebert Suconic <cl...@gmail.com>
wrote:
> For the use case you're after. (No hard syncs). Mmap is a good candidate.
> Probably better.
>
>
> Libaio was engineered the case where you hard sync with callbacks from the
> Linux os
> On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
>
> > hi clebertsuconic:
> > i read the blog
> > https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
> > By default Apache ActiveMQ Artemis will try and use an AIO journal.But it
> > seems like that Mmap is also a good implemention.which one gives more
> > performance?
> >
> >
> >
> > --
> > View this message in context:
> > http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-
> tp4726911p4726992.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
> --
> Clebert Suconic
>
Re: why AvtiveMq is slowly than Kafka?
Posted by Clebert Suconic <cl...@gmail.com>.
For the use case you're after. (No hard syncs). Mmap is a good candidate.
Probably better.
Libaio was engineered the case where you hard sync with callbacks from the
Linux os
On Sat, Jun 3, 2017 at 12:46 PM wangqinghuan <10...@qq.com> wrote:
> hi clebertsuconic:
> i read the blog
> https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
> By default Apache ActiveMQ Artemis will try and use an AIO journal.But it
> seems like that Mmap is also a good implemention.which one gives more
> performance?
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4726992.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
Clebert Suconic
Re: why AvtiveMq is slowly than Kafka?
Posted by wangqinghuan <10...@qq.com>.
hi clebertsuconic:
i read the blog
https://activemq.apache.org/artemis/docs/2.1.0/persistence.html
By default Apache ActiveMQ Artemis will try and use an AIO journal.But it
seems like that Mmap is also a good implemention.which one gives more
performance?
--
View this message in context: http://activemq.2283324.n4.nabble.com/why-AvtiveMq-is-slowly-than-Kafka-tp4726911p4726992.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.