[mina] Added more methods to ByteBuffer.

[Trustin Lee <tr...@gmail.com>]

Hi all,

Here's the list of the new methods:

* asInputStream() and asOutputStream() - this will improve the integration
with the legacy softwares.
* getPascalString() and putPascalString() - now you can read and write one
of the most popular string encoding in networking.
* getObject() and putObject() - you can read and write a Java object with no
sweat if you're too busy.  (though we still need an
ObjectSerializationFilter for those who're really busy. ;)

Any question and criticism is welcome.

Cheers,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/

Re: [mina] Added more methods to ByteBuffer.

[Trustin Lee <tr...@gmail.com>]

Hi Kaspar,

2005/11/29, Kaspar Lüthi <ka...@shinypixel.ch>:
>
> Trustin Lee wrote:
> >
> > * getPascalString() and putPascalString() - now you can read and write
> one
> > of the most popular string encoding in networking.
>
> ehm, as far as i know and read, the length of a pascal string is normally
> defined in a single byte, not two.


Oops, I have to change the name.  Any suggestion considering your additional
idea (i.e. padding and variable length field length)?

maybe you could overload the PascalString-methods with a parameter,
> which defines the length (number of bytes) of the length-header?
>
> the padding-thing might be to special to add to mina, but i wouldn't avoid
> using it if it's there :) could be a second parameter with a boolean flag
> (i.e.
> "doPadding").


Great idea.  But boolean parameter is not appropriate because the padding
can be four bytes at max in some protocols.

initially i wanted to convert all ByteBuffer to minas-ByteBuffer, but
> without
> duplicate() i can't see how.
>
> it's some kind of chat-server, so the server typically receives a message
> from a
> user and forwards it to all members in the same chat-group.
>
> that's where i thought, the duplicate()-method fits perfectly, since it
> does
> not copy the data itself but only creates private limit/capacity/position
> for
> the copy.


It is extremely hard to implement because MINA ByteBuffer has a reference
count field in itself.  So if we duplicate the buffer, it is hard to track
the acquirement and release of the reference count.  I don't know it is even
possible.  That's why I removed duplicate() method from MINA ByteBuffer.

i actually don't have a big idea how the mina-ByteBuffer works, but i
> assume,
> that session.write() does make a copy of the data, when wrapping the
> native
> buffer.


MINA ByteBuffer.wrap() doesn't copy the data, but the created buffer is not
go to the buffer pool when it is released by default.  You can let it go to
the pool by setting 'pooled' property, but it can cause memory leak, so you
have to be careful.

i actually don't see the difficulty of implementing duplicate() - can't you
> just
> increase the aquisition-counter in the pool, when duplicating?


No.  Things get really complex if user sets 'AutoExpand' property.  then the
internal reference to NIO buffer can change at any time.  Duplicated buffers
have to track this change in real time, and it will be a lot of overhead.

ok, this might be naive (haven't used java between jdk1.2 and jdk1.5...),
> and
> duplicate() is not necessary at all using the pooled mina-ByteBuffer. if
> this
> is the
> case, i would be glad, to have some short explanation why my assumptions
> are
> wrong.


Copying costs.  But we cannot implement duplicate method easily.  It would
be great if you have any good idea. :)

HTH,
Trustin

the server i made using mina, is live since 2 weeks with 0.9 snapshot. we
> serve about max. 600 concurrent users. haven't had any problem since.
> great
> work guys. for me it means a big relief from my own NIO-related troubles.


That's great!  :)
--
what we call human nature is actually human habit
--
http://gleamynode.net/

Re: [mina] Added more methods to ByteBuffer.

[Kaspar Lüthi <ka...@shinypixel.ch>]

hi

Trustin Lee wrote:
>
> * getPascalString() and putPascalString() - now you can read and write one
> of the most popular string encoding in networking.

ehm, as far as i know and read, the length of a pascal string is normally
defined in a single byte, not two.

i mention this, because i have another formatting wating for a convenience
method :)

http://www.macromedia.com/cfusion/knowledgebase/index.cfm?id=tn_15465
"kMoaMmValueType_String:
4 bytes with string length, then N bytes of string data, not null terminated,
but padded to an even byte boundary"

maybe you could overload the PascalString-methods with a parameter,
which defines the length (number of bytes) of the length-header?

the padding-thing might be to special to add to mina, but i wouldn't avoid
using it if it's there :) could be a second parameter with a boolean flag (i.e.
"doPadding").

---------

another thing i really missed with minas-ByteBuffer is the duplicate-method.

i adopted an existing project to mina (which was very straightforward and took
me less than a day, thank you), where i already did handle the data with
nio-ByteBufffers.

initially i wanted to convert all ByteBuffer to minas-ByteBuffer, but without
duplicate() i can't see how.

it's some kind of chat-server, so the server typically receives a message from a
user and forwards it to all members in the same chat-group.

that's where i thought, the duplicate()-method fits perfectly, since it does
not copy the data itself but only creates private limit/capacity/position for
the copy.

group:
public void sendMessage(ByteBuffer buffer) {
	for (int i = 0; i < usersArray.length; i++) {
		usersArray[i].sendMessage(buffer);
	}
}

user:
public void sendMessage(ByteBuffer buff) {
	session.write(buff.duplicate());
}

i actually don't have a big idea how the mina-ByteBuffer works, but i assume,
that session.write() does make a copy of the data, when wrapping the native
buffer.

is this assumption correct?

if yes, it looks like unnecessary overhead. imagine writing one identical message
(ByteBuffer) to 40+ sessions, it also uses more memory then.

On 4/19/05, Trustin Lee <tr...@gmail.com> wrote:
> There were also some changes due to introduction of auto expand property:
>
> * duplicate(), slice(), asReadOnlyBuffer() is disappeared; I don't see
> any easy way to implement this functionality, and it is not used in
> most cases especially when all buffers are pooled.

i actually don't see the difficulty of implementing duplicate() - can't you just
increase the aquisition-counter in the pool, when duplicating?

ok, this might be naive (haven't used java between jdk1.2 and jdk1.5...), and
duplicate() is not necessary at all using the pooled mina-ByteBuffer. if this
is the
case, i would be glad, to have some short explanation why my assumptions are
wrong.

greetings


p.s.
the server i made using mina, is live since 2 weeks with 0.9 snapshot. we
serve about max. 600 concurrent users. haven't had any problem since. great
work guys. for me it means a big relief from my own NIO-related troubles.