You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by samdowning <sa...@gmail.com> on 2015/04/08 12:16:49 UTC
Throughput drop with NIO transport as compared to TCP transport
Hi ActiveMQDevs,
I was fiddling around with NIO transport connector in ActiveMQ v5.11.1. I
can see the expected thread count reduction when using NIO connector. I did
some quick throughput testing also and found that with NIO transport
connector the throughput drops by more than 1.5 times to what I get with TCP
transport connector.
I used 1 subscriber and 1 producer on queue with non-persistent messages to
measure the throughput.
I tried the same thing 20 producers and 1 subscriber but similar results -
throughput with NIO was much less than TCP.
Is throughput expected to drop with NIO transport connector?
Thanks,
Sam
--
View this message in context: http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by Tim Bain <tb...@alumni.duke.edu>.
Hey Anuj: in that original thread, you saw only a small decrease in
throughput (53k to 47k), not 1.5x. Maybe you could help Sam figure out
what's different between his setup and yours...
Tim
On Apr 9, 2015 7:52 AM, "artnaseef" <ar...@artnaseef.com> wrote:
> Sorry - JMeter or custom Java code are only examples. Other contributions
> will be considered as well.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494p4694620.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by artnaseef <ar...@artnaseef.com>.
Sorry - JMeter or custom Java code are only examples. Other contributions
will be considered as well.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494p4694620.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by artnaseef <ar...@artnaseef.com>.
This is a great effort here.
Please consider sharing test tools. JMeter profiles or custom java code.
One thing I'm considering is capturing performance tests in another repo for
ActiveMQ, together with baselines (i.e. captured runs - preferably with a
solid capture of hardware configuration details and O/S level metrics
captured as well).
Anyway, if you can contribute to the same, that would be awesome.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494p4694618.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by oliverd <ol...@hotmail.com>.
it is not only about the number of consumers, producers
the question is how many consumers, producers actively send, receive data on
a connection
standard IO means one thread serves a connection, independent of the actual
usage - even if it's idle
NIO means decoupling - a thread is allocated (from a pool) on demand (if a
request needs to be served)
if you have 1000 consumers which actively transfer data then you will have
the same number of threads compared to blocking IO
what you want is that thread count does not increase in parallel to the
connection count (as this does not scale)
I would expect a slight overhead for NIO as the thread handling adds some
overhead (but this is the price for scaling) - given that the JVM NIO
throughput is comparable to blocking IO
--
View this message in context: http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494p4694615.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by Tim Bain <tb...@alumni.duke.edu>.
Kevin, Sam was actually part of that original thread.
Sam, as you probably remember from that discussion, Christian pointed out
that NIO uses a thread pool so you'll probably only see a decrease in
thread usage if you have more consumers (and more producers) than threads
in the thread pool, and I wonder if it takes those same types of conditions
to see the performance disadvantage disappear. I'd try like 100 of each
(at thecsame time) and see what the performance is like...
Tim
On Apr 9, 2015 1:40 AM, "samdowning" <sa...@gmail.com> wrote:
> Thanks for taking interest in this Kevin.
>
> As I already mentioned, I did those tests with 1 consumer, 1 producer and 1
> consumer, 20 producers.
>
> How many consumers and producers do you suggest I repeat the tests with?
>
> Thanks,
> Sam
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494p4694595.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by samdowning <sa...@gmail.com>.
Thanks for taking interest in this Kevin.
As I already mentioned, I did those tests with 1 consumer, 1 producer and 1
consumer, 20 producers.
How many consumers and producers do you suggest I repeat the tests with?
Thanks,
Sam
--
View this message in context: http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494p4694595.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by Kevin Burton <bu...@spinn3r.com>.
How many connections are you trying this with? I suspect it might have to
do with NIO only using one of your cores.
On Wed, Apr 8, 2015 at 6:12 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> looks like others see a throughput drop:
>
>
> http://activemq.2283324.n4.nabble.com/NIO-transport-performance-for-ActiveMQ-td4670977.html
>
> On Wed, Apr 8, 2015 at 6:10 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
>> Interesting. Would be nice to see if there were any benchmarks for this.
>>
>> On Wed, Apr 8, 2015 at 3:16 AM, samdowning <sa...@gmail.com>
>> wrote:
>>
>>> Hi ActiveMQDevs,
>>>
>>> I was fiddling around with NIO transport connector in ActiveMQ v5.11.1. I
>>> can see the expected thread count reduction when using NIO connector. I
>>> did
>>> some quick throughput testing also and found that with NIO transport
>>> connector the throughput drops by more than 1.5 times to what I get with
>>> TCP
>>> transport connector.
>>>
>>> I used 1 subscriber and 1 producer on queue with non-persistent messages
>>> to
>>> measure the throughput.
>>> I tried the same thing 20 producers and 1 subscriber but similar results
>>> -
>>> throughput with NIO was much less than TCP.
>>>
>>> Is throughput expected to drop with NIO transport connector?
>>>
>>> Thanks,
>>> Sam
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>
>>
>>
>> --
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> … or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>> <http://spinn3r.com>
>>
>>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by Kevin Burton <bu...@spinn3r.com>.
looks like others see a throughput drop:
http://activemq.2283324.n4.nabble.com/NIO-transport-performance-for-ActiveMQ-td4670977.html
On Wed, Apr 8, 2015 at 6:10 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> Interesting. Would be nice to see if there were any benchmarks for this.
>
> On Wed, Apr 8, 2015 at 3:16 AM, samdowning <sa...@gmail.com>
> wrote:
>
>> Hi ActiveMQDevs,
>>
>> I was fiddling around with NIO transport connector in ActiveMQ v5.11.1. I
>> can see the expected thread count reduction when using NIO connector. I
>> did
>> some quick throughput testing also and found that with NIO transport
>> connector the throughput drops by more than 1.5 times to what I get with
>> TCP
>> transport connector.
>>
>> I used 1 subscriber and 1 producer on queue with non-persistent messages
>> to
>> measure the throughput.
>> I tried the same thing 20 producers and 1 subscriber but similar results -
>> throughput with NIO was much less than TCP.
>>
>> Is throughput expected to drop with NIO transport connector?
>>
>> Thanks,
>> Sam
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Throughput drop with NIO transport as compared to TCP transport
Posted by Kevin Burton <bu...@spinn3r.com>.
Interesting. Would be nice to see if there were any benchmarks for this.
On Wed, Apr 8, 2015 at 3:16 AM, samdowning <sa...@gmail.com> wrote:
> Hi ActiveMQDevs,
>
> I was fiddling around with NIO transport connector in ActiveMQ v5.11.1. I
> can see the expected thread count reduction when using NIO connector. I did
> some quick throughput testing also and found that with NIO transport
> connector the throughput drops by more than 1.5 times to what I get with
> TCP
> transport connector.
>
> I used 1 subscriber and 1 producer on queue with non-persistent messages to
> measure the throughput.
> I tried the same thing 20 producers and 1 subscriber but similar results -
> throughput with NIO was much less than TCP.
>
> Is throughput expected to drop with NIO transport connector?
>
> Thanks,
> Sam
>
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Throughput-drop-with-NIO-transport-as-compared-to-TCP-transport-tp4694494.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>