Does mina use scattered writes?

["Luke Hubbard (luke@codegent.com)" <ki...@gmail.com>]

Hi Guys,

First post to the list, but I'm a long time mina fan. We are using it in the
red5 server with great success. I'm currently refactoring the event design
and trying to figure ways to reduce overhead. I have a question about using
lots of small buffers and making many writes.

Let me explain the situation.

We have implemented RTMP protocol, which is a proprietary streaming protocol
used by flash. Data in the protocol is chunked into 128 byte segments, in
between each chunk there are a variable number of header bytes. The headers
range in size from 1 byte to 12. The data can be audio, video, or amf
serialized objects.

Currently we have an internal buffer in the packet which holds the raw data,
in the serializer this is copied to another buffer adding the header bytes.
This process has to be done for every client the packet is written to, since
the header bytes may change for different clients.

I'm wondering if I could split this up, making lots of writes.. header
buffer, chunk buffer, header buffer, chunk buffer, etc. This way I can share
the chunk buffers between clients (by caching them against the packet), and
dispose them when we have finished writing the packet to all clients. I
imagine this would be more memory efficient. Video packets can get quite
large.

So.. my questions are..

Will making lots of writes have a negative impact on performance?
Or is it better to use a larger buffer even if it cant be shared?
And... does mina use scattered writes under the hood?

Hope the above makes sense, and you can point me in the right direction.
Thanks for your work on an excellent framework.

-- Luke

Re: Does mina use scattered writes?

[Trustin Lee <tr...@gmail.com>]

On 4/16/06, robert.j.greig@jpmorgan.com <ro...@jpmorgan.com> wrote:
>
> I made some modifications to the SocketIoProcessor to take advantage of
> this but on a 100Mb network could not observe any improvement. We now have a
> Gb network in our test environment and some more tests so I plan on
> revisiting this at some point.
>

Your plan sounds very exciting. Please give us an update when a test on a
gigabit network is done if you don't mind. :)

Thanks,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41  4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4  455E 1C62 A7DC 0255 ECA6

Re: Does mina use scattered writes?

[Vinod Panicker <vi...@gmail.com>]

Hi,

On 4/14/06, Luke Hubbard (luke@codegent.com) <ki...@gmail.com> wrote:
> Hi Guys,

<snip/>

> Will making lots of writes have a negative impact on performance?

Theoretically, it should.  Practically, if you switch off TCP_NODELAY,
the NIO frameword should be able to handle it better.  Another problem
you might have in this case is the packet header and the actual
payload coming in independent tcp packets.  I assume that its alright
with you if that happens.

> Or is it better to use a larger buffer even if it cant be shared?

A zero-copy mechanism is most efficient.  Maybe the read-only buffers
would be of help in this case.  I'm not sure about that though -
others would be able to give better suggestions.

> And... does mina use scattered writes under the hood?

AFAIK the NIO library doesn't support this.  Mustang is bringing quite
a few improvements, including support for epoll.  That would be a
blast!

> Hope the above makes sense, and you can point me in the right direction.
> Thanks for your work on an excellent framework.

HTH,
Vinod.