You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Ronny Standtke <Ro...@gmx.net> on 2007/11/04 11:46:03 UTC
Comparison with NIO Framework
Hi all,
Unaware of MINA I also implemented a very simple NIO framework:
http://nioframework.sourceforge.net
After I announced it on the Sun Developer Forums someone pointed me to MINA.
Therefore I wrote some simple benchmarks to compare MINA with my framework
and got surprisingly bad results with MINA, see here:
http://forum.java.sun.com/thread.jspa?threadID=5213915&tstart=0
So far I only got feedback from Esmont Pitt who criticized MINA.
But I still can not believe that MINA is really _THAT_ slow. Maybe you have
a hint for me where I can "pull some stops" so that the benchmark runs
faster with MINA.
Thanks
Ronny Standtke
--
View this message in context: http://www.nabble.com/Comparison-with-NIO-Framework-tf4746048s16868.html#a13571619
Sent from the Apache MINA Support Forum mailing list archive at Nabble.com.
Re: Comparison with NIO Framework
Posted by Trustin Lee <tr...@gmail.com>.
On 11/6/07, Ronny Standtke <Ro...@gmx.net> wrote:
> Trustin Lee wrote:
> >
> > I'm not sure you configured the Echo server properly. Setting the
> > receiveBufferSize to 3000 instantly boosted the throughput to 50MB/s.
> > I don't know why, but it seems like the receiveBufferSize is affecting
> > the performance seriously.
> >
> This is very strange. Today I run some tests with different
> receiveBufferSize values and up to a size of EXACTLY 10837 Byte I only get
> 86 KByte/s. Starting from 10838 I get around 50 MByte/s.
>
> My system is a P4 running Debian testing with Java 1.6.0_05-ea (Yes, the
> early access release because 1.6.0u3 still has a serious NIO bug I run into,
> see here: http://forum.java.sun.com/thread.jspa?threadID=5135128 Sun
> Developer Forum )
>
> After enlarging the receiveBufferSize a little more (30k) I suddenly get the
> following exception on the MINA side:
>
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:633)
> at
> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
> at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:57)
> at sun.nio.ch.IOUtil.read(IOUtil.java:205)
> at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
> at
> org.apache.mina.transport.socket.nio.SocketIoProcessor.read(SocketIoProcessor.java:208)
> at
> org.apache.mina.transport.socket.nio.SocketIoProcessor.process(SocketIoProcessor.java:188)
> at
> org.apache.mina.transport.socket.nio.SocketIoProcessor.access$400(SocketIoProcessor.java:44)
> at
> org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:464)
> at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
> at java.lang.Thread.run(Thread.java:619)
>
> Ouch! Do I have to turn another knob somewhere in MINA to prevent this
> Exception?
It seems like the echo server example in 1.x is somewhat broken.
Today, I was able to get 190MiB/s with the latest snapshot though.
Thank you again for your help. BenchmarkClient was very helpful to
me.
> Trustin Lee wrote:
> >
> > I was able to fix a bug thanks to your report. Thank you so much! If
> > you have some free time, you could check out the latest MINA from our
> > source code repository and try the test again.
> >
>
> Thank you very much for this offer. I will take a look for sure.
> Unfortunately my time is very limited right now, so this may take a while...
No problem. Please take your time.
Cheers,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Comparison with NIO Framework
Posted by Ronny Standtke <Ro...@gmx.net>.
Trustin Lee wrote:
>
> I'm not sure you configured the Echo server properly. Setting the
> receiveBufferSize to 3000 instantly boosted the throughput to 50MB/s.
> I don't know why, but it seems like the receiveBufferSize is affecting
> the performance seriously.
>
This is very strange. Today I run some tests with different
receiveBufferSize values and up to a size of EXACTLY 10837 Byte I only get
86 KByte/s. Starting from 10838 I get around 50 MByte/s.
My system is a P4 running Debian testing with Java 1.6.0_05-ea (Yes, the
early access release because 1.6.0u3 still has a serious NIO bug I run into,
see here: http://forum.java.sun.com/thread.jspa?threadID=5135128 Sun
Developer Forum )
After enlarging the receiveBufferSize a little more (30k) I suddenly get the
following exception on the MINA side:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at
java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:57)
at sun.nio.ch.IOUtil.read(IOUtil.java:205)
at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
at
org.apache.mina.transport.socket.nio.SocketIoProcessor.read(SocketIoProcessor.java:208)
at
org.apache.mina.transport.socket.nio.SocketIoProcessor.process(SocketIoProcessor.java:188)
at
org.apache.mina.transport.socket.nio.SocketIoProcessor.access$400(SocketIoProcessor.java:44)
at
org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:464)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Thread.java:619)
Ouch! Do I have to turn another knob somewhere in MINA to prevent this
Exception?
Trustin Lee wrote:
>
> I was able to fix a bug thanks to your report. Thank you so much! If
> you have some free time, you could check out the latest MINA from our
> source code repository and try the test again.
>
Thank you very much for this offer. I will take a look for sure.
Unfortunately my time is very limited right now, so this may take a while...
Greetings & Thanks
Ronny
--
View this message in context: http://www.nabble.com/Comparison-with-NIO-Framework-tf4746048s16868.html#a13596140
Sent from the Apache MINA Support Forum mailing list archive at Nabble.com.
Re: Comparison with NIO Framework
Posted by Trustin Lee <tr...@gmail.com>.
I found Ronny's BenchmarkClient and Server very useful for measuring
the maximum throughput and tuning SocketIoProcessor. We could start
from there.
Cheers,
Trustin
On 11/5/07, Niklas Therning <ni...@trillian.se> wrote:
> Fedechicco wrote:
> > I'm not such a great writer on this list, but I'm an affectioned fan of
> > MINA, and I think that if you can show the evolution of the performance as
> > well of the functionalities, MINA would grow its (already fantastic)
> > credibility.
> >
> > I'm totally for the standard benchmark idea!
> >
> > also that way people could get an idea of how much performance bottlenecks
> > are due to MINA or too the user-defined structure over it.
> >
> >
> I agree that it would be very useful to have a standard benchmark test
> suite, especially to avoid degrading performance when we implement new
> features. I think we already have this in JIRA:
>
> https://issues.apache.org/jira/browse/DIRMINA-56
>
> --
> Niklas Therning
> www.spamdrain.net
>
>
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Comparison with NIO Framework
Posted by Niklas Therning <ni...@trillian.se>.
Fedechicco wrote:
> I'm not such a great writer on this list, but I'm an affectioned fan of
> MINA, and I think that if you can show the evolution of the performance as
> well of the functionalities, MINA would grow its (already fantastic)
> credibility.
>
> I'm totally for the standard benchmark idea!
>
> also that way people could get an idea of how much performance bottlenecks
> are due to MINA or too the user-defined structure over it.
>
>
I agree that it would be very useful to have a standard benchmark test
suite, especially to avoid degrading performance when we implement new
features. I think we already have this in JIRA:
https://issues.apache.org/jira/browse/DIRMINA-56
--
Niklas Therning
www.spamdrain.net
Re: Comparison with NIO Framework
Posted by Fedechicco <fe...@gmail.com>.
I'm not such a great writer on this list, but I'm an affectioned fan of
MINA, and I think that if you can show the evolution of the performance as
well of the functionalities, MINA would grow its (already fantastic)
credibility.
I'm totally for the standard benchmark idea!
also that way people could get an idea of how much performance bottlenecks
are due to MINA or too the user-defined structure over it.
Re: Comparison with NIO Framework
Posted by Michael Bauroth <mi...@falcom.de>.
Great idea! Would be a good thing to selftest the own changes against
the values reached before. Additionally I like MINA for it's chain
concept and the several (increasing number) of "plugins", so that it
would be bad, if the people choose different frameworks only because
they are quicker.
Regards
Michael
Emmanuel Lecharny schrieb:
> Hi Trustin, and team !
>
> This comparison thread is really interesting. As many people out there
> are wondering what kind of performances can be reached with MINA, in
> various aspects ( throughput, number of users, etc), it would be
> interesting to have a set of perfomance scenario defined and run against
> different versions of MINA.
>
> Exposing those numbers for each new version can help developper to know
> if what they have added to the server has impact on performance (I was
> specifically thinging about the 'logger' thread two or three weeks ago
> ). It can also expose the real numbers, with correct settings, instead
> of what people find just by playing there own scenario on there own
> computers.
>
> Last, not least, a performance scenario with sources is very valuable to
> people who want to understand how to get the best practice when writing
> a server using MINA.
>
> wdyt ?
>
Re: Comparison with NIO Framework
Posted by Emmanuel Lecharny <el...@gmail.com>.
Hi Trustin, and team !
This comparison thread is really interesting. As many people out there
are wondering what kind of performances can be reached with MINA, in
various aspects ( throughput, number of users, etc), it would be
interesting to have a set of perfomance scenario defined and run against
different versions of MINA.
Exposing those numbers for each new version can help developper to know
if what they have added to the server has impact on performance (I was
specifically thinging about the 'logger' thread two or three weeks ago
). It can also expose the real numbers, with correct settings, instead
of what people find just by playing there own scenario on there own
computers.
Last, not least, a performance scenario with sources is very valuable to
people who want to understand how to get the best practice when writing
a server using MINA.
wdyt ?
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: Comparison with NIO Framework
Posted by Trustin Lee <tr...@gmail.com>.
On 11/4/07, Trustin Lee <tr...@gmail.com> wrote:
<snip/>
> > After I announced it on the Sun Developer Forums someone pointed me to MINA.
> > Therefore I wrote some simple benchmarks to compare MINA with my framework
> > and got surprisingly bad results with MINA, see here:
> > http://forum.java.sun.com/thread.jspa?threadID=5213915&tstart=0
>
> I'm not sure you configured the Echo server properly. Setting the
> receiveBufferSize to 3000 instantly boosted the throughput to 50MB/s.
> I don't know why, but it seems like the receiveBufferSize is affecting
> the performance seriously.
>
> However, your BenchmarkServer showed around 170MB/s in my machine.
> It's very impressive, and I'd love to run some profiler on MINA.
I was able to fix a bug thanks to your report. Thank you so much! If
you have some free time, you could check out the latest MINA from our
source code repository and try the test again.
I got around 150MB/s in my machine with MINA and 190MB/s with your
framework. This means the MINA team needs to tune MINA more, although
both numbers are pretty good for most network applications.
Thanks,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Comparison with NIO Framework
Posted by Trustin Lee <tr...@gmail.com>.
Hi Ronny,
Thanks for posting the message to the forum first of all. I usually
don't monitor Sun forum. The Sun form was down, so I've just seen the
post.
On 11/4/07, Ronny Standtke <Ro...@gmx.net> wrote:
> Hi all,
>
> Unaware of MINA I also implemented a very simple NIO framework:
> http://nioframework.sourceforge.net
I am glad to see another NIO framework. It's good to have more
choices that fits on various situations. :)
> After I announced it on the Sun Developer Forums someone pointed me to MINA.
> Therefore I wrote some simple benchmarks to compare MINA with my framework
> and got surprisingly bad results with MINA, see here:
> http://forum.java.sun.com/thread.jspa?threadID=5213915&tstart=0
I'm not sure you configured the Echo server properly. Setting the
receiveBufferSize to 3000 instantly boosted the throughput to 50MB/s.
I don't know why, but it seems like the receiveBufferSize is affecting
the performance seriously.
However, your BenchmarkServer showed around 170MB/s in my machine.
It's very impressive, and I'd love to run some profiler on MINA.
> So far I only got feedback from Esmont Pitt who criticized MINA.
I wonder Esmont actually used MINA because:
1) MINA buffer doesn't imply any copy. You can always wrap an
existing NIO buffer.
2) You can always disable the default thread model whenever you want.
That's too bad that he didn't post his idea on MINA design issues
directly to this forum though.
Thanks again,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: Comparison with NIO Framework
Posted by John Preston <by...@gmail.com>.
Can you post the results.
John
On 11/4/07, Ronny Standtke <Ro...@gmx.net> wrote:
>
> Hi all,
>
> Unaware of MINA I also implemented a very simple NIO framework:
> http://nioframework.sourceforge.net
>
> After I announced it on the Sun Developer Forums someone pointed me to MINA.
> Therefore I wrote some simple benchmarks to compare MINA with my framework
> and got surprisingly bad results with MINA, see here:
> http://forum.java.sun.com/thread.jspa?threadID=5213915&tstart=0
> So far I only got feedback from Esmont Pitt who criticized MINA.
>
> But I still can not believe that MINA is really _THAT_ slow. Maybe you have
> a hint for me where I can "pull some stops" so that the benchmark runs
> faster with MINA.
>
> Thanks
>
> Ronny Standtke
> --
> View this message in context: http://www.nabble.com/Comparison-with-NIO-Framework-tf4746048s16868.html#a13571619
> Sent from the Apache MINA Support Forum mailing list archive at Nabble.com.
>
>