[GSoC] MINA performance tuning platform

[Alex Karasulu <ak...@apache.org>]

Hi students and potential MINA mentors,

I had a couple conversations with a few peeps that wished they had a month
or two to write a slick new framework for load testing MINA based
applications while using MINA itself to write the framework.  My experiences
with MINA and various crappy tools out there make me realize that
performance tuning a network application is half the battle.  I know others
probably feel the same.

Then all this GSoC stuff is going on and the idea occurs to one of us (not
me it was Emmanuel's idea) that we can make this a nice tight discrete
project where a student can accomplish this task while learning about MINA
and getting involved in our community.  This is just a win win situation.

The idea is simple.  We have a set of load injecting worker processes that
are coordinated by managers on each load injecting host.  The managers
simply control the life cycle of worker processes and get their orders to do
so from a data collecting and collating server with a management console.
This requires some kind of simple control protocol which can be written
using MINA.  For the console I dream of a slick GWT interface where you can
launch tests and control injector hosts.  Oh the fun you can have with
statistics etc ... anyways back from dream land ... Ideally separate
managers reside in a set of load injecting machines and the system under
test resides on yet another.  Likewise the console server can run on a
separate machine as well.  The clients funnel data to the collating server
for jobs and are network applications that likewise can be written using
MINA.  Regardless of the topology used this is the general model presented
by may performance testing frameworks like Grinder and SLAMD.

The neat thing is some frameworks can even start running optimizations where
parameters are varied to find optimal running conditions.  I'd like to do
this too but perhaps also have an agent in the tested application as a
Filter probably to both monitor and influence the application.  This makes
the server more of a white box than a black box undergoing testing from the
outside.  It could for example vary standard MINA server parameters to find
optimal settings while monitoring server process information like memory and
thread usage etc.  There are many possibilities here.  We could even use
Genetic Algorithms and Optimization Algorithms to control and influence the
optimization process - ok getting carried away.  For starters of course this
feature may not be present :D.  However the idea is the student can take
this as far as they like.

The bottom line is this is a great project for a student interested in using
MINA to tune MINA while getting their feet wet and involved in this
community.  MINA users will now have a framework they can use and hopefully
contribute back to.  Everyone is happy.

Thoughts?

Anyone interested in mentoring or taking on this project as a student in
GSoC?

Thanks,
Alex

Re: [GSoC] MINA performance tuning platform

["이희승 (Trustin Lee)" <tr...@gmail.com>]

I also would like to see Mike to come up with his proposal, which is
also going to be exciting.

2008-03-11 (화), 21:10 -0400, Alex Karasulu 쓰시길:
> Hi students and potential MINA mentors,
> 
> I had a couple conversations with a few peeps that wished they had a month
> or two to write a slick new framework for load testing MINA based
> applications while using MINA itself to write the framework.  My experiences
> with MINA and various crappy tools out there make me realize that
> performance tuning a network application is half the battle.  I know others
> probably feel the same.
> 
> Then all this GSoC stuff is going on and the idea occurs to one of us (not
> me it was Emmanuel's idea) that we can make this a nice tight discrete
> project where a student can accomplish this task while learning about MINA
> and getting involved in our community.  This is just a win win situation.
> 
> The idea is simple.  We have a set of load injecting worker processes that
> are coordinated by managers on each load injecting host.  The managers
> simply control the life cycle of worker processes and get their orders to do
> so from a data collecting and collating server with a management console.
> This requires some kind of simple control protocol which can be written
> using MINA.  For the console I dream of a slick GWT interface where you can
> launch tests and control injector hosts.  Oh the fun you can have with
> statistics etc ... anyways back from dream land ... Ideally separate
> managers reside in a set of load injecting machines and the system under
> test resides on yet another.  Likewise the console server can run on a
> separate machine as well.  The clients funnel data to the collating server
> for jobs and are network applications that likewise can be written using
> MINA.  Regardless of the topology used this is the general model presented
> by may performance testing frameworks like Grinder and SLAMD.
> 
> The neat thing is some frameworks can even start running optimizations where
> parameters are varied to find optimal running conditions.  I'd like to do
> this too but perhaps also have an agent in the tested application as a
> Filter probably to both monitor and influence the application.  This makes
> the server more of a white box than a black box undergoing testing from the
> outside.  It could for example vary standard MINA server parameters to find
> optimal settings while monitoring server process information like memory and
> thread usage etc.  There are many possibilities here.  We could even use
> Genetic Algorithms and Optimization Algorithms to control and influence the
> optimization process - ok getting carried away.  For starters of course this
> feature may not be present :D.  However the idea is the student can take
> this as far as they like.
> 
> The bottom line is this is a great project for a student interested in using
> MINA to tune MINA while getting their feet wet and involved in this
> community.  MINA users will now have a framework they can use and hopefully
> contribute back to.  Everyone is happy.
> 
> Thoughts?
> 
> Anyone interested in mentoring or taking on this project as a student in
> GSoC?
> 
> Thanks,
> Alex
-- 
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/

Re: [GSoC] MINA performance tuning platform

[Emmanuel Lecharny <el...@gmail.com>]

Hi,

Fedechicco wrote:
> Ok, I imagine that the "load injection" is to be done using a simple
> protocol, like an echo or something, only to test the network capabilities
> of the whole thing, am I right?
>   
Well, not only. We don't really want to test the network capabilities, 
as it won't reflect the real usage of MINA. As MINA is a framework 
allowing codec to be plugged-in, the most interesting part of the load 
test would be to test the plugged codec.

To test MINA capabilities, we will just use an empty codec.

So basically, the injectors should be able to use any kind of protocol.

For instance, let say you want to load-test LDAP protocol, you will want 
to use a client which will inject LDAP PDU to the server. The generic 
client  will be a simple transport system, the data being injected are 
generated by something which is external to the injector (like a 
plugin). The code would be something like :

int inject( dataProducer )
{
  // Get the data to be injected
  bytes = dataProducer.getData()
  t0 = current time
  send( bytes )
  t1 = current time
  return t1 - t0
}

assuming you are interested in the time taken to send the data. As you 
can see, the dataProducer is not known by the injector. It's up to the 
tester to write this producer, accordingly to what he wants to test.

Does it make sense ?

The important part of the work is to synchronize the different injectors 
in order to get a realistic load charge (ie, if you have 2 injectors on 
2 different machines which are not running in the same time, then you 
won't test the load)

-- 
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org

Re: [GSoC] MINA performance tuning platform

[Alex Karasulu <ak...@apache.org>]

On Fri, Mar 14, 2008 at 7:00 AM, Fedechicco <fe...@gmail.com> wrote:

> Ok, I imagine that the "load injection" is to be done using a simple
> protocol, like an echo or something, only to test the network capabilities
> of the whole thing, am I right?


No that would be the protocol under test implemented by the MINA
application.  Besides this the client needs a simple protocol to send back
it's statistics to the console.  Let's call this the statistics protocol.
This protocol would be very simple like the echo protocol.


>
> Or we are trying to do something using a generic already implemented
> protocol?


The protocol being tested can be custom designed or can be an implementation
of a specified protocol like HTTP, LDAP, FTP etc.


> I mean something like: I have some new MINA server and MINA client
> for my new protocol, and simply with a tool like a filter I can test the
> performance of my server. It has to be something like that?
>

Well you need more than just a filter in the server under test.  This
performance filter will make the MINA server under test into a white box but
we still need statistics coming in from those clients.  The info from he
filter should be correlated with the info from the clients by the console.


>
> I think I'm over with questions, I almost understood.
>
>
Cool.

Alex

Re: [GSoC] MINA performance tuning platform

[Fedechicco <fe...@gmail.com>]

Ok, I imagine that the "load injection" is to be done using a simple
protocol, like an echo or something, only to test the network capabilities
of the whole thing, am I right?
Or we are trying to do something using a generic already implemented
protocol? I mean something like: I have some new MINA server and MINA client
for my new protocol, and simply with a tool like a filter I can test the
performance of my server. It has to be something like that?

I think I'm over with questions, I almost understood.

Fed

Re: [GSoC] MINA performance tuning platform

[Alex Karasulu <ak...@apache.org>]

On Wed, Mar 12, 2008 at 6:08 PM, Fedechicco <fe...@gmail.com> wrote:

> Hi!
> I am a student, I didn't quite understood exactly the work to,


The idea is to build a load testing platform for MINA based network
applications.  You have clients on different hosts acting as load
injectors.  They make requests against the MINA application under test.
These clients collect information about rates and send it to a central
console server that collects this information and collates them.  Users can
schedule tests and see the results from the console.


> More over I'm studying by myself some Genetic Algorithm implementations, I
> used it for neural networks but it shouldn't be too much different.


You probably will not have time for this at first.  The biggest part of this
is building the clients and console etc.  But if you'd like to use that for
parameter optimization go right ahead.


> So, I would be glad to participate, but I can't say I really understood
> the
> job 8)
>
>
HTH,
Alex

Re: [GSoC] MINA performance tuning platform

[Fedechicco <fe...@gmail.com>]

Hi!
I am a student, I didn't quite understood exactly the work to, but I'm a
great fan of MINA as I used it ( v0.8 ) for some server I developed some
time ago, and I would be glad of helping in some way.
More over I'm studying by myself some Genetic Algorithm implementations, I
used it for neural networks but it shouldn't be too much different.
So, I would be glad to participate, but I can't say I really understood the
job 8)

Federico Bonelli

Re: [GSoC] MINA performance tuning platform

[Maarten Bosteels <mb...@gmail.com>]

Hello Alex,

Have been dreaming of such tool as well ...
I just wish I was still a student, with an ocean of time :-)

You clearly wrote the description with a lot of enthusiasm.
I really hope someone accepts the challenge.

Maarten

On Wed, Mar 12, 2008 at 2:10 AM, Alex Karasulu <ak...@apache.org> wrote:

> Hi students and potential MINA mentors,
>
> I had a couple conversations with a few peeps that wished they had a month
> or two to write a slick new framework for load testing MINA based
> applications while using MINA itself to write the framework.  My
> experiences
> with MINA and various crappy tools out there make me realize that
> performance tuning a network application is half the battle.  I know
> others
> probably feel the same.
>
> Then all this GSoC stuff is going on and the idea occurs to one of us (not
> me it was Emmanuel's idea) that we can make this a nice tight discrete
> project where a student can accomplish this task while learning about MINA
> and getting involved in our community.  This is just a win win situation.
>
> The idea is simple.  We have a set of load injecting worker processes that
> are coordinated by managers on each load injecting host.  The managers
> simply control the life cycle of worker processes and get their orders to
> do
> so from a data collecting and collating server with a management console.
> This requires some kind of simple control protocol which can be written
> using MINA.  For the console I dream of a slick GWT interface where you
> can
> launch tests and control injector hosts.  Oh the fun you can have with
> statistics etc ... anyways back from dream land ... Ideally separate
> managers reside in a set of load injecting machines and the system under
> test resides on yet another.  Likewise the console server can run on a
> separate machine as well.  The clients funnel data to the collating server
> for jobs and are network applications that likewise can be written using
> MINA.  Regardless of the topology used this is the general model presented
> by may performance testing frameworks like Grinder and SLAMD.
>
> The neat thing is some frameworks can even start running optimizations
> where
> parameters are varied to find optimal running conditions.  I'd like to do
> this too but perhaps also have an agent in the tested application as a
> Filter probably to both monitor and influence the application.  This makes
> the server more of a white box than a black box undergoing testing from
> the
> outside.  It could for example vary standard MINA server parameters to
> find
> optimal settings while monitoring server process information like memory
> and
> thread usage etc.  There are many possibilities here.  We could even use
> Genetic Algorithms and Optimization Algorithms to control and influence
> the
> optimization process - ok getting carried away.  For starters of course
> this
> feature may not be present :D.  However the idea is the student can take
> this as far as they like.
>
> The bottom line is this is a great project for a student interested in
> using
> MINA to tune MINA while getting their feet wet and involved in this
> community.  MINA users will now have a framework they can use and
> hopefully
> contribute back to.  Everyone is happy.
>
> Thoughts?
>
> Anyone interested in mentoring or taking on this project as a student in
> GSoC?
>
> Thanks,
> Alex
>