You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Marc Boorshtein <mb...@gmail.com> on 2007/02/23 21:26:06 UTC
MINA 1.0.2 Scalability?
I was curious if any scalability testing has been done on MINA based
servers and any scalability tips that may be available. I've already
upped my SocketAcceptor threads which is not seeming to help. MyVD
uses MINA for it's protocol stack (based on the apacheds stack). No
matter what I do, I can't seem to go beyond 10 concurrent connections
and was wondering if anyone else had seen this?
Thanks
Marc
Re: MINA 1.0.2 Scalability?
Posted by Alex Karasulu <ak...@apache.org>.
That's odd we can do many concurrent connections with ApacheDS well beyond
10. At least this is what I have encountered.
Alex
On 2/23/07, Marc Boorshtein <mb...@gmail.com> wrote:
>
> I was curious if any scalability testing has been done on MINA based
> servers and any scalability tips that may be available. I've already
> upped my SocketAcceptor threads which is not seeming to help. MyVD
> uses MINA for it's protocol stack (based on the apacheds stack). No
> matter what I do, I can't seem to go beyond 10 concurrent connections
> and was wondering if anyone else had seen this?
>
> Thanks
> Marc
>
Re: MINA 1.0.2 Scalability?
Posted by Marc Boorshtein <mb...@gmail.com>.
On 2/24/07, Michael Grundvig <mi...@electrotank.com> wrote:
> I've been meaning to post this for a while but have been lazy. With Mr.
> Royal's help, a new project my company has been working on has been able to
> achieve 120,000 concurrent connections with MINA.
>
Well, that just gives me the warm and fuzzies for choosing MINA :-D
my own tests have shown VERY good performance thus far. While my
tests aren't truly conclusive yet between them and the testimonials
here I am confident I made the rite choice.
Marc
Re: MINA 1.0.2 Scalability?
Posted by Alex Karasulu <ak...@apache.org>.
Michael,
On 2/24/07, Michael Grundvig <mi...@electrotank.com> wrote:
>
> Absolutely Alex, as soon as the server goes public I'll be glad to provide
> a
> testimonial for MINA.
Cool thanks!
Additionally, we will be producing a rather
> substantial set of load testing results and posting them online. They
> include environment configurations, OS/hardware specs, etc. We are also
> giving away the load tester source code we used for the application. It's
> quite specific to our server but might help someone else out.
Have you had a chance to look at SLAMD? Here's the URL just in case:
http://slamd.com
It's a nice peice of software providing several useful stats on performance
etc. Although designed for LDAP it can work for just about anything with
some customizations.
I will say this, generating that much load takes more machines then the
> server system itself.
No doubt. I bet your server sucks wire now tho :).
> It's also a tricky piece of code. Anyone making a load
> generator should plan way ahead if they want it to work properly. We also
> ran into some very strange problems.
Yeah that's why I recommend slamd. It correlates the total thru-put by
correlating the data gathered from all clients. It helps get a good
accurate estimate of just how many requests per second a server can manage.
Regards,
Alex
Re: MINA 1.0.2 Scalability?
Posted by Michael Grundvig <mi...@electrotank.com>.
Absolutely Alex, as soon as the server goes public I'll be glad to provide a
testimonial for MINA. Additionally, we will be producing a rather
substantial set of load testing results and posting them online. They
include environment configurations, OS/hardware specs, etc. We are also
giving away the load tester source code we used for the application. It's
quite specific to our server but might help someone else out.
I will say this, generating that much load takes more machines then the
server system itself. It's also a tricky piece of code. Anyone making a load
generator should plan way ahead if they want it to work properly. We also
ran into some very strange problems. Most network developers know about file
descriptor limits but what about ephemeral ports? MINA works so well that we
had to raise the ephemeral port limit on the load generators to prevent us
from running out of outbound ports! For details:
http://www.ncftp.com/ncftpd/doc/misc/ephemeral_ports.html
Thanks!
Mike
----- Original Message -----
From: "Alex Karasulu" <ak...@apache.org>
To: <de...@mina.apache.org>
Sent: Saturday, February 24, 2007 8:39 AM
Subject: Re: MINA 1.0.2 Scalability?
> Wow this is great Michael. I think this is a great candidate for our
> testimonials section. WDYT?
>
> Alex
>
> On 2/24/07, Michael Grundvig <mi...@electrotank.com> wrote:
>>
>> I've been meaning to post this for a while but have been lazy. With Mr.
>> Royal's help, a new project my company has been working on has been able
>> to
>> achieve 120,000 concurrent connections with MINA.
>>
>> We are using a distributed model that looks something like this: client
>> -->
>> "gateway" --> server where many gateways front-end the server. By
>> offsetting
>> the connection overhead to gateways, we have been able to scale much
>> farther
>> then normal. Depending on the number of messages per second, it's
>> possible
>> to scale even farther. We believe a single server could handle 1/4
>> million
>> users or more without any changes (assuming you had the appropriate
>> number
>> of gateways). In the end, garbage collector churn has been the limiting
>> factor in every test. Once we hit 50,000+ inbound messages per second on
>> the
>> server any garbage collector delays become a bottleneck from which we
>> can't
>> always recover. Work is being done to correct this though by reducing
>> object
>> churn.
>>
>> MINA has been the single largest reason we were able to create the
>> application at all. Simply put, MINA works beautifully with an elegant
>> and
>> powerful design. A big thank you goes out from me to the MINA developers
>> and
>> contributors.
>>
>> Thanks!
>>
>> Mike
>>
>> P.S. When the server goes public (it's a purchasable product) I'll go
>> ahead
>> and publish the final load test results. Right now we are working on
>> making
>> it so servers can be clustered together. This will raise the user limit
>> potentially into the millions. MINA rocks!
>>
>> ----- Original Message -----
>> From: "Marc Boorshtein" <mb...@gmail.com>
>> To: <de...@mina.apache.org>; "Apache Directory Developers List"
>> <de...@directory.apache.org>
>> Sent: Friday, February 23, 2007 2:26 PM
>> Subject: MINA 1.0.2 Scalability?
>>
>>
>> >I was curious if any scalability testing has been done on MINA based
>> > servers and any scalability tips that may be available. I've already
>> > upped my SocketAcceptor threads which is not seeming to help. MyVD
>> > uses MINA for it's protocol stack (based on the apacheds stack). No
>> > matter what I do, I can't seem to go beyond 10 concurrent connections
>> > and was wondering if anyone else had seen this?
>> >
>> > Thanks
>> > Marc
>>
>>
>
Re: MINA 1.0.2 Scalability?
Posted by Alex Karasulu <ak...@apache.org>.
Wow this is great Michael. I think this is a great candidate for our
testimonials section. WDYT?
Alex
On 2/24/07, Michael Grundvig <mi...@electrotank.com> wrote:
>
> I've been meaning to post this for a while but have been lazy. With Mr.
> Royal's help, a new project my company has been working on has been able
> to
> achieve 120,000 concurrent connections with MINA.
>
> We are using a distributed model that looks something like this: client
> -->
> "gateway" --> server where many gateways front-end the server. By
> offsetting
> the connection overhead to gateways, we have been able to scale much
> farther
> then normal. Depending on the number of messages per second, it's possible
> to scale even farther. We believe a single server could handle 1/4 million
> users or more without any changes (assuming you had the appropriate number
> of gateways). In the end, garbage collector churn has been the limiting
> factor in every test. Once we hit 50,000+ inbound messages per second on
> the
> server any garbage collector delays become a bottleneck from which we
> can't
> always recover. Work is being done to correct this though by reducing
> object
> churn.
>
> MINA has been the single largest reason we were able to create the
> application at all. Simply put, MINA works beautifully with an elegant and
> powerful design. A big thank you goes out from me to the MINA developers
> and
> contributors.
>
> Thanks!
>
> Mike
>
> P.S. When the server goes public (it's a purchasable product) I'll go
> ahead
> and publish the final load test results. Right now we are working on
> making
> it so servers can be clustered together. This will raise the user limit
> potentially into the millions. MINA rocks!
>
> ----- Original Message -----
> From: "Marc Boorshtein" <mb...@gmail.com>
> To: <de...@mina.apache.org>; "Apache Directory Developers List"
> <de...@directory.apache.org>
> Sent: Friday, February 23, 2007 2:26 PM
> Subject: MINA 1.0.2 Scalability?
>
>
> >I was curious if any scalability testing has been done on MINA based
> > servers and any scalability tips that may be available. I've already
> > upped my SocketAcceptor threads which is not seeming to help. MyVD
> > uses MINA for it's protocol stack (based on the apacheds stack). No
> > matter what I do, I can't seem to go beyond 10 concurrent connections
> > and was wondering if anyone else had seen this?
> >
> > Thanks
> > Marc
>
>
Re: MINA 1.0.2 Scalability?
Posted by Michael Grundvig <mi...@electrotank.com>.
I've been meaning to post this for a while but have been lazy. With Mr.
Royal's help, a new project my company has been working on has been able to
achieve 120,000 concurrent connections with MINA.
We are using a distributed model that looks something like this: client -->
"gateway" --> server where many gateways front-end the server. By offsetting
the connection overhead to gateways, we have been able to scale much farther
then normal. Depending on the number of messages per second, it's possible
to scale even farther. We believe a single server could handle 1/4 million
users or more without any changes (assuming you had the appropriate number
of gateways). In the end, garbage collector churn has been the limiting
factor in every test. Once we hit 50,000+ inbound messages per second on the
server any garbage collector delays become a bottleneck from which we can't
always recover. Work is being done to correct this though by reducing object
churn.
MINA has been the single largest reason we were able to create the
application at all. Simply put, MINA works beautifully with an elegant and
powerful design. A big thank you goes out from me to the MINA developers and
contributors.
Thanks!
Mike
P.S. When the server goes public (it's a purchasable product) I'll go ahead
and publish the final load test results. Right now we are working on making
it so servers can be clustered together. This will raise the user limit
potentially into the millions. MINA rocks!
----- Original Message -----
From: "Marc Boorshtein" <mb...@gmail.com>
To: <de...@mina.apache.org>; "Apache Directory Developers List"
<de...@directory.apache.org>
Sent: Friday, February 23, 2007 2:26 PM
Subject: MINA 1.0.2 Scalability?
>I was curious if any scalability testing has been done on MINA based
> servers and any scalability tips that may be available. I've already
> upped my SocketAcceptor threads which is not seeming to help. MyVD
> uses MINA for it's protocol stack (based on the apacheds stack). No
> matter what I do, I can't seem to go beyond 10 concurrent connections
> and was wondering if anyone else had seen this?
>
> Thanks
> Marc
Re: MINA 1.0.2 Scalability?
Posted by jian wu <he...@gmail.com>.
Hi Marc,
> This was in fact the problem. MINA is scaling great! Thanks for the
> quick testimonials. It made it much easier for me to debug the issue.
Great to know that you found the problem. MINA is indeed scaling great:-)
Best regards,
Jian
On 2/23/07, Marc Boorshtein <mb...@gmail.com> wrote:
> On 2/23/07, Marc Boorshtein <mb...@gmail.com> wrote:
> > thanks for the testimony. the problem is most likely pebkac :-D
> > (problem exists between keyboard and chair).
> >
> > Marc
> >
>
> This was in fact the problem. MINA is scaling great! Thanks for the
> quick testimonials. It made it much easier for me to debug the issue.
>
> Thanks
> Marc
>
Re: MINA 1.0.2 Scalability?
Posted by Marc Boorshtein <mb...@gmail.com>.
On 2/23/07, Marc Boorshtein <mb...@gmail.com> wrote:
> thanks for the testimony. the problem is most likely pebkac :-D
> (problem exists between keyboard and chair).
>
> Marc
>
This was in fact the problem. MINA is scaling great! Thanks for the
quick testimonials. It made it much easier for me to debug the issue.
Thanks
Marc
Re: MINA 1.0.2 Scalability?
Posted by Marc Boorshtein <mb...@gmail.com>.
On 2/23/07, Marc Boorshtein <mb...@gmail.com> wrote:
> thanks for the testimony. the problem is most likely pebkac :-D
> (problem exists between keyboard and chair).
>
> Marc
>
This was in fact the problem. MINA is scaling great! Thanks for the
quick testimonials. It made it much easier for me to debug the issue.
Thanks
Marc
Re: MINA 1.0.2 Scalability?
Posted by Marc Boorshtein <mb...@gmail.com>.
thanks for the testimony. the problem is most likely pebkac :-D
(problem exists between keyboard and chair).
Marc
Re: MINA 1.0.2 Scalability?
Posted by jian wu <he...@gmail.com>.
Hi Marc,
I had developed an in-house MINA based network application
for routing application level protocol, which is Text Protocol
based on TCP/IP. We were using Mina 0.94 and is migrating
to Mina 1.0.1.
For my L&P testing, I developed a Simulator using Python
2.5 with Twisted 2.5 and I also developed Multi-Threaded
Test Driver using Python 2.5.
Usually, I run two Test Driver Instances on two separate
machines each one will launch 20 current threads to hit the
Server on another machine which'll route all the requests to
the Simulators and forward the responses back to the Test
Driver as client.
I can run my test driver over night and over weekend, I don't
see any problem Mina handling 40 concurrent client threads.
We are using Solaris 10 x86 boxes for these kind of testing.
Thanks,
Jian
On 2/23/07, Marc Boorshtein <mb...@gmail.com> wrote:
> I was curious if any scalability testing has been done on MINA based
> servers and any scalability tips that may be available. I've already
> upped my SocketAcceptor threads which is not seeming to help. MyVD
> uses MINA for it's protocol stack (based on the apacheds stack). No
> matter what I do, I can't seem to go beyond 10 concurrent connections
> and was wondering if anyone else had seen this?
>
> Thanks
> Marc
>