You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Vinod Panicker <vi...@gmail.com> on 2005/03/28 13:24:23 UTC
Update on MINA test
Just wanted to update you guys on the latest.
100K+ concurrent connections reached.
Setup -
- Commented notifyIdleSessions()
- Using latest MINA release 159225
- MINA based single-threaded server
- Blocking I/O based single-threaded client
- Connections fired @ 40/sec
- JVM options -Xmn500M -Xms1000M -Xmx1000M
- Server on Windows 2003 Server - AthlonXP 2400+ (2 Ghz), 1 GB RAM
- Clients on Windows 2003 Server, Windows 2000 Server and Linux 2.6.9-1.667
- Session read buffer size set to 8 bytes
Results -
- Total concurrent connections - 103432
- JVM mem usage - 541M
- JVM VM size - 1.5G
- JVM file handles used - 311,392
- CPU usage after reaching max connections - 30% (consistent)
- CPU usage during test - 100% (after around 5000 connections)
- Connections getting accepted per second degrade but not in a linear
manner (around 100K they were going at around 20/sec but sometimes at
around 2/sec)
Total system wide file handles used - 421919
Total system wide mem usage - 2GB
Current problems observed
- CPU usage during test is consistently 100%
- Memory usage still seems to be on the high side
- Connector takes 100% CPU - blocking I/O based client uses around 4%
- notifyIdleSessions() not being used
Current plan
- To try same test with non-MINA based NIO server and client
- To try same test with non-MINA based Blocking I/O server and client
Basically I want to put a finger on the CPU usage problem.
Also, does any one know what the latest sun jdk (1.5.0_02) uses as the
impl for select() on linux? Is it select(), poll() or epoll() ?
Regards,
Vinod.
Re: [Mina] IllegalStateException: Already released buffer
Posted by Jan Andersson <ja...@minq.se>.
> Oops, it was a bug. I checked in the fix.
Thanks!
12 minutes from reporting the problem to a checked in fix.
That's not to bad ;)
/Janne
Re: [Mina] IllegalStateException: Already released buffer
Posted by Trustin Lee <tr...@gmail.com>.
Hi Janne,
On Tue, 29 Mar 2005 15:55:52 +0200, Jan Andersson <ja...@minq.se> wrote:
> We get exceptions with the latest version of Mina in our application:
>
> Exception in thread "SocketIoProcessor"
> java.lang.IllegalStateException: Already released buffer.
> at
> org.apache.mina.common.ByteBuffer.acquire(ByteBuffer.java:148)
> at
> org.apache.mina.io.socket.SocketIoProcessor.read(SocketIoProcessor.java:
> 240)
> at
> org.apache.mina.io.socket.SocketIoProcessor.processSessions(SocketIoProc
> essor.java:221)
> at
> org.apache.mina.io.socket.SocketIoProcessor.access$200(SocketIoProcessor
> .java:41)
> at
> org.apache.mina.io.socket.SocketIoProcessor$Worker.run(SocketIoProcessor
> .java:496)
>
> I suspect that this is caused by me trying to set the receive buffer
> size
> from new default size of 1K to 8K.
>
> What I do is that I call
> SocketSessionConfig.setSessionReceiveBufferSize(8192)
> from my IoHandler.sessionOpened() method like this:
>
> public void sessionOpened(IoSession ioSession)
> {
> SocketSessionConfig ssc = (SocketSessionConfig)
> session.getConfig();
> ssc.setSessionReceiveBufferSize(8192);
> ...
> }
>
> Obviously this isn't the right way to change the receive buffer size,
> but where/how
> would be the correct way to set this?
Oops, it was a bug. I checked in the fix. Sorry for the inconvenience!
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
[Mina] IllegalStateException: Already released buffer
Posted by Jan Andersson <ja...@minq.se>.
We get exceptions with the latest version of Mina in our application:
Exception in thread "SocketIoProcessor"
java.lang.IllegalStateException: Already released buffer.
at
org.apache.mina.common.ByteBuffer.acquire(ByteBuffer.java:148)
at
org.apache.mina.io.socket.SocketIoProcessor.read(SocketIoProcessor.java:
240)
at
org.apache.mina.io.socket.SocketIoProcessor.processSessions(SocketIoProc
essor.java:221)
at
org.apache.mina.io.socket.SocketIoProcessor.access$200(SocketIoProcessor
.java:41)
at
org.apache.mina.io.socket.SocketIoProcessor$Worker.run(SocketIoProcessor
.java:496)
I suspect that this is caused by me trying to set the receive buffer
size
from new default size of 1K to 8K.
What I do is that I call
SocketSessionConfig.setSessionReceiveBufferSize(8192)
from my IoHandler.sessionOpened() method like this:
public void sessionOpened(IoSession ioSession)
{
SocketSessionConfig ssc = (SocketSessionConfig)
session.getConfig();
ssc.setSessionReceiveBufferSize(8192);
...
}
Obviously this isn't the right way to change the receive buffer size,
but where/how
would be the correct way to set this?
/Janne
Re: Update on MINA test
Posted by Berin Loritsch <bl...@d-haven.org>.
Vinod Panicker wrote:
> Just wanted to update you guys on the latest.
>
> 100K+ concurrent connections reached.
Awsome!
> Also, does any one know what the latest sun jdk (1.5.0_02) uses as the
> impl for select() on linux? Is it select(), poll() or epoll() ?
Not without downloading source code for the VM and library. I don't
want to do that though because I am working on something that could be
viewed as competing with the JVM. Better not to cause issues there.
If you have a Java developer login, you could pull it down yourself.
Of course they do have some forum that you might be able to ask. Not
sure where it is located though.
Re: Update on MINA test
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Monday 28 March 2005 21:46, Vinod Panicker wrote:
> Do we have
> some kind of svn "lab" concept here where I can put the files?
You could also put it into the DPML lab, since anyone here can easily get an
account over at DPML, whilst the opposite is not true.
Both Steve and I are closely monitoring (without interfering :o) ) the
ApacheDS project, and quite happy that you are helping them out.
Cheers
Niclas
Re: Update on MINA test
Posted by Alex Karasulu <ao...@bellsouth.net>.
Vinod Panicker wrote:
>On Mon, 28 Mar 2005 22:04:26 +0900, Trustin Lee <tr...@gmail.com> wrote:
>
>
>>Hi,
>>
>>Thanks Vinod. Keep up the good work! :)
>>
>>BTW, it would be nice if we can test together. How's test suite going?
>>
>>
>>
>
>Till now its just been simple java files that do all this. Still dont
>have an *integrated* suite. Will get that done tomorrow. Do we have
>some kind of svn "lab" concept here where I can put the files?
>
>Need to first decide on what exactly its gonna be. Pointers welcome.
>
>
Trustin, Vinod,
We might want to create an area in the sandbox where Vinod can keep his
sources. Either that or we could make the network trunk a multiproject
and add his code to a perfsuite maven project. Your choice either way.
However keep in mind ... if adding to network trunk testcases that run
should not be ones that actually test perf bringing user boxes to their
knees. Rather the test cases in a perfsuite should unit test persuite
code. The perfsuite itself should do the intense tests when run outside
of tests. This way when a user builds the mina code from the trunk with
a maven multiproject:install they build the suite but not necessarily
run it against mina.
Alex
Re: Update on MINA test
Posted by Vinod Panicker <vi...@gmail.com>.
On Mon, 28 Mar 2005 22:04:26 +0900, Trustin Lee <tr...@gmail.com> wrote:
> Hi,
>
> Thanks Vinod. Keep up the good work! :)
>
> BTW, it would be nice if we can test together. How's test suite going?
>
Till now its just been simple java files that do all this. Still dont
have an *integrated* suite. Will get that done tomorrow. Do we have
some kind of svn "lab" concept here where I can put the files?
Need to first decide on what exactly its gonna be. Pointers welcome.
Regards,
Vinod.
Re: Update on MINA test
Posted by Trustin Lee <tr...@gmail.com>.
Hi,
Thanks Vinod. Keep up the good work! :)
BTW, it would be nice if we can test together. How's test suite going?
Thanks,
Trustin
On Mon, 28 Mar 2005 17:41:36 +0530, Vinod Panicker <vi...@gmail.com> wrote:
> OK.. tried with the following setup now -
>
> Setup
>
> - non-MINA based NIO server
> - non-MINA based NIO client
> - 40 connections/sec
> - 50K connections targeted
> - rest same as before
>
> Results
>
> - 50K connections
> - CPU usage behaviour *exactly* same as MINA - 100% CPU after around
> 5K connections
> - NIO based client very nice on CPU - 2-4%
> - CPU usage @ 50K concurrent connections consistent at 18%
> - JVM mem usage - 342M
> - JVM vm size - 1.2G
> - JVM File handles used - 150,666
>
> Observations
>
> - MINA doesn't seem to add any overheads on the server side
> - NIO seems to have some probs with select(). Note that the client,
> which is also having the same number of concurrent connections and is
> NIO based, uses hardly any cpu during the test, and practically none
> at the end. Since the client is not using select(), it seems that
> most CPU time is getting chewed up in there somewhere. Also, the
> previous observations on Linux showed that most of the CPU time was
> being used by the system during the test run (forgot to see it after a
> successful test).
>
> TODO
>
> - Run test with blocking I/O server (client irrelevant since no probs out there)
> - Run NIO server with profiler
> - Run NIO test on server to find out system/user cpu usage
>
> Any one know how to find out how much system/user cpu an app is taking
> on windows?
>
> Regards,
> Vinod.
>
--
what we call human nature is actually human habit
--
http://gleamynode.net/
Re: Update on MINA test
Posted by Vinod Panicker <vi...@gmail.com>.
On Mon, 28 Mar 2005 14:57:50 +0200, Stephane Bailliez
<sb...@apache.org> wrote:
> Vinod Panicker wrote:
>
> >Thanks Stephane for the info. Will search for an epoll based impl.
> >Or else seems like i'll have to get drunk and start with my own
> >provider
> >
> >
> Apparently the blackdown guy do have a epoll provider within their JDK.
> It can be a start.
>
I remember that blackdown had epoll support in their 1.4 jdk. AFAIK
they still have to provide it for 1.5. Lemme dig there some more.
Regards,
Vinod.
Re: Update on MINA test
Posted by Stephane Bailliez <sb...@apache.org>.
Vinod Panicker wrote:
>Thanks Stephane for the info. Will search for an epoll based impl.
>Or else seems like i'll have to get drunk and start with my own
>provider
>
>
Apparently the blackdown guy do have a epoll provider within their JDK.
It can be a start.
Re: Update on MINA test
Posted by Vinod Panicker <vi...@gmail.com>.
Latest update with blocking i/o server
Setup -
- Blocking I/O based server
- NIO based non-MINA client
- 40 connections/sec
- 50K connections targeted
- rest same as before
Results
- CPU usage extremely nice at around 2-4% with occasional spikes to
10% during test run
- CPU usage with 50K concurrent sessions - 0%
- JVM mem usage - 54M (!!!)
- JVM vm size - 1G (due to startup options)
- JVM file handles - 150,246
Observations
- Absolutely no stress on the system. CPU usage hardly anything. At
0% frequently during test as well.
- Memory usage much much lower than NIO based server
- CPU usage after test run 0%. No system overhead whatsoever
- Seems like there's somethings seriously wrong inside NIO.
Thanks Stephane for the info. Will search for an epoll based impl.
Or else seems like i'll have to get drunk and start with my own
provider
TODO
- Run profiler on NIO based server
- Check on available epoll impl's
- SCREAM
Regards,
Vinod.
Re: Update on MINA test
Posted by Stephane Bailliez <sb...@apache.org>.
Vinod Panicker wrote:
>Any one know how to find out how much system/user cpu an app is taking
>on windows?
>
>
You have the Management console (mmc) or directly the performance
monitor tool (perfmon).
You can normally access it from the 'Administrative tools' or directly
from the command line.
Re: Update on MINA test
Posted by Vinod Panicker <vi...@gmail.com>.
OK.. tried with the following setup now -
Setup
- non-MINA based NIO server
- non-MINA based NIO client
- 40 connections/sec
- 50K connections targeted
- rest same as before
Results
- 50K connections
- CPU usage behaviour *exactly* same as MINA - 100% CPU after around
5K connections
- NIO based client very nice on CPU - 2-4%
- CPU usage @ 50K concurrent connections consistent at 18%
- JVM mem usage - 342M
- JVM vm size - 1.2G
- JVM File handles used - 150,666
Observations
- MINA doesn't seem to add any overheads on the server side
- NIO seems to have some probs with select(). Note that the client,
which is also having the same number of concurrent connections and is
NIO based, uses hardly any cpu during the test, and practically none
at the end. Since the client is not using select(), it seems that
most CPU time is getting chewed up in there somewhere. Also, the
previous observations on Linux showed that most of the CPU time was
being used by the system during the test run (forgot to see it after a
successful test).
TODO
- Run test with blocking I/O server (client irrelevant since no probs out there)
- Run NIO server with profiler
- Run NIO test on server to find out system/user cpu usage
Any one know how to find out how much system/user cpu an app is taking
on windows?
Regards,
Vinod.
Re: Update on MINA test
Posted by Stephane Bailliez <sb...@apache.org>.
Vinod Panicker wrote:
>Also, does any one know what the latest sun jdk (1.5.0_02) uses as the
>impl for select() on linux? Is it select(), poll() or epoll() ?
>
>
Apparently it is poll() for Linux. (I don't have the src for the _02 but
I guess they did not change that)
For Solaris there seems to be 2 implementations, one use /dev/poll (
sun.nio.ch.DevPollSelectorProvider ) and the other uses poll()
(sun.nio.ch.PollSelectorProvider) , the factory
(sun.nio.ch.DefaultSelectorProvider) is selecting the /dev/poll version
for os.version for y >= 7 (Solaris 7+ ?) when the os version is 'x.y'.
Note that you can use the system property
'java.nio.channels.spi.SelectorProvider=<classname>' to select the
provider or use the META-INF/services with the same key to select
another provider (you can do your own if you're motivated :o)