DIRMINA-301 New Multi threaded SocketIOProcessor to improve fairness of socket reads/writes

[Martin Ritchie <ri...@apache.org>]

Hi,

I'm a member of the Apache Qpid development team. Qpid, which is a
message broker, uses Mina for our transport layer.

Some of our use cases involve reading a writing from the same socket,
e.g. sending a stream of messages that are acknowledged individually
by the broker. In our tests, we have seen poor throughput and
starvation in some cases. The current mina SocketIOProcessor is biased
to writing data and has no capability to read and write to a given
socket concurrently.

We have created a multi threaded SocketIOProcessor that allows reads
and write to occur on different threads. The use of this
SocketIOProcessor has result in an improvement in overall throughput
and reduced memory problems under heavy read/writes per session. I
have created a JIRA (DIRMINA-301) which contains a standalone test
that compares the two SocketIOProcessors.

It would be great of some of you with existing mina applications could
test this new SocketIOProcessor and see if there is difference in
performance.

The test shows a difference in speed between the IOProcessors.

The test can also be used to demonstrate the problems with
writeRequestQueue that causes an out of memory problem on the client
when sending is not fast enough.

I can see a references to the writeRequestQueue problem in JIRAs:
 DIRMINA-144 Traffic shaping filter
 DIRMINA-206 Prevent OutOfMemoryError when a server or a client is overloaded
 DIRMINA-262 Controlling rate of writes in Mina 0.8.2

The traffic shapping filter would ensure a particular data rate from
the applicatoin however, the client code calling mina also needs to be
throttled. So that the unbounded writequeue does not use all the
available memory. This has been raised as JIRA DIRMINA-302.

Cheers
-- 
Martin Ritchie