[MINA] What about a cluster for network load balancing?

[fe...@gmail.com]

I was thinking about several MINA working on different jvms, each of theese
connected with a single MINA thinking structure that do the protocol
specified work.

The connection between the thinking-block and the network-farm should be
massively compressed, and the protocol that should be used for the
connection between the two blocks should implements broadcast (ie a method 
like "writeToAll(Object tosend)" in a SessionGroup class).

The thinking-block use the MINAs of network-farm as if they are in the same
jvm, using acceptors and connectors, binding the ports on the remote MINAs
IP addresses as if they are some local IP addresses, on local interfaces.

This can be usefull for each protocol that must receive one message and
resend it over all the other connections (or a large part of them), because
the final clients can connect to one of the network-farm MINAs
indifferently, like if ther's a single server listening on different
interfaces of the same pc.
The broadcast messages between thinking-block and network-farm guarantee the
efficency needed.

I developed this same cluster model for a chat-like server system (Direct 
Connect protocol), and it
works greatly, but I didn't know MINA yet, so this was implemented in a
disgusting manner on a disgusting network framework, and it is only for one
specific protocol, I'd like to have this implemented for any server
application, transparently on MINA.

It can be more easy implemented if mina provide the SessionGroup class with 
the broadcast method writeToAll(), or something like that.

Do you like the idea?
Could it be usefull?
what do you think about that structure?
It can be another communication transport supported by MINA or should it be 
another alone project?

Hi all!  by Fed

Found RC1 Binaries

[Ole Ersoy <ol...@yahoo.com>]

Never mind on the RC1 Binaries Question.

I see all the mirrors are updated now.

There's even an RPM!  I'm impressed!

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [MINA] What about a cluster for network load balancing?

[Fedechicco <fe...@gmail.com>]

I have done that stuff, now it works like an IoAcceptor, and it works in
this way:

On the "network-farm" side:

there is an IoHandler (called ForwarderHandler) that need to be connected to
the thinking-block in some way ( by now with some Connector, but it can be
done with an Acceptor too ).
After a "centralConnection" is opened ( the connection to the
thinking-block), the ForwarderHandler will open the tcp ports required by
the thinking block, and then it will forward all the events (receive, close,
open, idle) of every new connection to the thinking block.
And this is all the story of the "network-farm" side.

On the "thinking-block" side:

I developed a IoAcceptor (called ForwarderAcceptor) that is a normal
acceptor on wich you can do the bind() and unbind() methods, as usual,
giving your own IoHandler (that I'll call GenericHandler) and setting your
own filterChains ( I copyed all this stuff from SocketAcceptor ).

ForwarderAcceptor have to talk with the network-farm side, and it does this
task with an IoHandler that works out the messages from and to the
network-farm.
ForwarderAcceptor have a method that return the IoHandler I have just talked
about, so you can do some little hack on it to do interesting thing, like a
tree structure of network-farms ( I'll explain it better ).
ForwarderAcceptor receives the connections of the network-farm programs and
receive from them the messages about the sessions handled.
And after all it forward this events to the GenericHandler, passing through
the appropriate chains.

And this is the end of the implementation.

Some features:
both IoHandler (the ForwarderHandler and the ForwarderAcceptorHandler) use
the same messages that are objects called ForwarderMessage.
Both them use ForwarderCodecFactory to translate messages into ByteBuffers.
In this way it is simple and performant to have a VmPipe connection between
ForwarderAcceptor and ForwarderHandler, you have only to not use the codec
at all.

By now it is not tested with more than one connection, and it do not provide
compression of data or the methods "writeToAll" or "writeToSessionGroup", so
by now it is just an "encapsuler", and by now it do not handle in the right
way the ByteBuffer acquire and release, so it doesn't work in VmPipe mode
because it throws IllegalStateExceptions, I have to work more on that.

It works on the lower level I can imagine, directly with ByteBuffers, so it
can be used to forward every kind of protocol.

Well, this is more or less all the news I can give you about this stuff, I'm
going to test it by my self with my applications, but I have to ask you
something.

There are news about SessionGroup implementation? Because I need it to do
the writeToAll methods, and I don't know how to manage the groups otherwise.

If somebody want to see the code I have written just ask me and I'll put it
on jira, but it is just a stub right now.

Re: [MINA] What about a cluster for network load balancing?

[fe...@gmail.com]

Ok Trustin, I'll try.
I have not the necessary experience to do another transport type by my own, so I'll not implement a complete MINA api transport, I'll just try to develop an example of the protocol.
So if somebody, who know mina better than me, likes the idea, then he can develop the transport type.

hi everybody!

Re: [MINA] What about a cluster for network load balancing?

[Trustin Lee <tr...@gmail.com>]

2006/1/19, fedechicco@gmail.com <fe...@gmail.com>:
>
> Hi Trustin,
> I am developing a basical solution that can be used from TCP only
> applications.
> I'm developing that dividing since now the protocol that I use to make the
> various parts communicate, and the higher level that manage the internal
> logic of the solution.
> So when everything will be ready we can easily change the protocol with
> the one we want, and by now I had written down a simple string based
> protocol, not so performant but easy to debug and sniff to.
> For my purpouse it would be really usefull the coming-soon "SessionGroup"
> class, with the method writeToAll(), because these is the kind of messages
> in wich this structure will be more performant.
>
> If you don't mind it I can do a SimpleSessionGroup interface that I'll use
> in this project, with the only method of addSession, removeSession and
> writeToAll().
>
> Will you mind it?
> WDYT?
>

Who'll mind it? :D

But please implement java.util.Set interface and provide additional
add/remove method which accepts IoSession as a parameter.

Thanks,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C

Re: [MINA] What about a cluster for network load balancing?

[fe...@gmail.com]

Hi Trustin,
I am developing a basical solution that can be used from TCP only applications.
I'm developing that dividing since now the protocol that I use to make the various parts communicate, and the higher level that manage the internal logic of the solution.
So when everything will be ready we can easily change the protocol with the one we want, and by now I had written down a simple string based protocol, not so performant but easy to debug and sniff to.
For my purpouse it would be really usefull the coming-soon "SessionGroup" class, with the method writeToAll(), because these is the kind of messages in wich this structure will be more performant.

If you don't mind it I can do a SimpleSessionGroup interface that I'll use in this project, with the only method of addSession, removeSession and writeToAll().

Will you mind it?
WDYT?

Re: [MINA] What about a cluster for network load balancing?

[Trustin Lee <tr...@gmail.com>]

Hi Fedechicco,

2006/1/7, fedechicco@gmail.com <fe...@gmail.com>:
>
> I was thinking about several MINA working on different jvms, each of
> theese
> connected with a single MINA thinking structure that do the protocol
> specified work.
>
> The connection between the thinking-block and the network-farm should be
> massively compressed, and the protocol that should be used for the
> connection between the two blocks should implements broadcast (ie a method
> like "writeToAll(Object tosend)" in a SessionGroup class).
>
> The thinking-block use the MINAs of network-farm as if they are in the
> same
> jvm, using acceptors and connectors, binding the ports on the remote MINAs
> IP addresses as if they are some local IP addresses, on local interfaces.
>
> This can be usefull for each protocol that must receive one message and
> resend it over all the other connections (or a large part of them),
> because
> the final clients can connect to one of the network-farm MINAs
> indifferently, like if ther's a single server listening on different
> interfaces of the same pc.
> The broadcast messages between thinking-block and network-farm guarantee
> the
> efficency needed.
>
> I developed this same cluster model for a chat-like server system (Direct
> Connect protocol), and it
> works greatly, but I didn't know MINA yet, so this was implemented in a
> disgusting manner on a disgusting network framework, and it is only for
> one
> specific protocol, I'd like to have this implemented for any server
> application, transparently on MINA.
>
> It can be more easy implemented if mina provide the SessionGroup class
> with
> the broadcast method writeToAll(), or something like that.
>
> Do you like the idea?
> Could it be usefull?
> what do you think about that structure?
> It can be another communication transport supported by MINA or should it
> be
> another alone project?


I agree with Alex.  This is really cool idea, but I'm not an expert in this
area.  Perhaps you could show us the way to go.

Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
PGP Key ID: 0x854B996C

Re: [MINA] What about a cluster for network load balancing?

[fe...@gmail.com]

"Alex Karasulu" <ao...@bellsouth.net> wrote:

> That's neat experiment.  I'd be interested in seeing a simple application 
> and some performance metrics.  I think TCP based protocols would most 
> benefit from this.

As I said I have implemented yet a server that use this kind of work, and I 
can give you the performance of my server right now:
I'm running only a single mina program as network-farm (i've called this 
program "dislocator", and that protocol "dislocation protocol"), but the 
results are impressive:
BandWidth Speed (out): 2,46 Kb/s

BandWidth Speed (in): 1,23 Kb/s

BandWidth Speed (total): 3,70 Kb/s


theese up are the bandwidth usages of the thinking-block

and theese down are the bandwidth usages of my single dislocator:
Users: 1160 ReadBand: 22,68 Kb/s writeBand: 377,96 Kb/s

You can see that there are 1160 users connected to this MINAs dislocator, so 
many many messages that must be sent to all users (it's a p2p protocol, they 
are the search messages) are sent from thinking-block to the network-farm 
only once, than my single dislocator duplicate it 1160 times and resend it 
over all the sockets.

You can see that it work well because this 
"must-to-be-sent-equal-to-all-the-users" messages are the higher part of the 
traffic.

> I think TCP based protocols would most benefit from this.


I agree, actually I've developed it for tcp only, but also UDP can work fine 
in this way.

>
> It sounds very interesting.  As with many ideas things come down to doing 
> some experimentation and testing out the theory.  I know that for example 
> SEDA sounds great on paper but can create so many headaches in practice.
>

Well, I've thought a lot about that project, and I know that it has a 
principal limitation: it can work fine only for protocol that have a poor 
CPU usage for protocol elaboration (the network load is divide between many 
jvms, but all the CPU work is done by the thinking-block), and that it can 
work well only for protocol like hubs, in wich a message must to be sent 
over many sockets, or must to be sent the same many times equal (in this 
case the an adaptive Huffman encode should be fine)

Every kind of protocol that do not have theese features will not have any 
bandwidth advantage from that system, but this kind of protocols are the 
most diffused chat - gaming - p2p protocols.

Re: [MINA] What about a cluster for network load balancing?

[Alex Karasulu <ao...@bellsouth.net>]

fedechicco@gmail.com wrote:

> I was thinking about several MINA working on different jvms, each of 
> theese
> connected with a single MINA thinking structure that do the protocol
> specified work.

That sick! I like it.  Definitely a more advanced way to develope a high 
throughput protocol server out of multiple nodes.

> The connection between the thinking-block and the network-farm should be
> massively compressed, and the protocol that should be used for the
> connection between the two blocks should implements broadcast (ie a 
> method like "writeToAll(Object tosend)" in a SessionGroup class).

There is probably a cut off were the simple model is better: meaning 
where network latency cost/overhead is offset by load.

> The thinking-block use the MINAs of network-farm as if they are in the 
> same
> jvm, using acceptors and connectors, binding the ports on the remote 
> MINAs
> IP addresses as if they are some local IP addresses, on local interfaces.
>
> This can be usefull for each protocol that must receive one message and
> resend it over all the other connections (or a large part of them), 
> because
> the final clients can connect to one of the network-farm MINAs
> indifferently, like if ther's a single server listening on different
> interfaces of the same pc.

Right that's really a neat result of this architecture.  You're not 
limited by the number of sockets on a single node.

> I developed this same cluster model for a chat-like server system 
> (Direct Connect protocol), and it
> works greatly, but I didn't know MINA yet, so this was implemented in a
> disgusting manner on a disgusting network framework, and it is only 
> for one
> specific protocol, I'd like to have this implemented for any server
> application, transparently on MINA.

That's neat experiment.  I'd be interested in seeing a simple 
application and some performance metrics.  I think TCP based protocols 
would most benefit from this.

> Do you like the idea?

It sounds very interesting.  As with many ideas things come down to 
doing some experimentation and testing out the theory.  I know that for 
example SEDA sounds great on paper but can create so many headaches in 
practice.

Alex