You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by 이희승 (Trustin Lee) <t...@gmail.com> on 2008/04/29 04:20:51 UTC
A quick prototyping of ByteArray interface
Hi folks,
I tried a quick prototyping of the ByteArray interface I've mentioned in
our previous discussion thread. I wanted to check in the code right now
for easier review, but ASF-wide SVN write access has been disabled due
to some hardware issue. Let me paste it here:
http://pastebin.com/m37fb44ba
The idea is to implement a ByteBuffer-backed ByteArray class and
implement some variable length buffer-of-arrays using it. WDYT?
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
Re: A quick prototyping of ByteArray interface
Posted by 이희승 (Trustin Lee) <t...@gmail.com>.
We are continuing our discussion about composite buffer in a different
thread. We decided to stick to ByteBuffer way, so my prototype is going
to burn in a trash bin. :)
Instead, I've proposed to introduce a new type called
CompositeByteBuffer. I think you might like that.
Rich Dougherty wrote:
> Yes, it has taken me a while to get that CompositeIoBuffer written. :-)
>
> In any case it looks like the discussion has moved on past needing a
> composite interface. But since I've already sketched something up I may
> as well post it anyway!
>
> http://pastebin.com/f3c9c5f8d
>
> I've cut a lot of corners, but hopefully the idea is still clear.
>
> In particular the Cursor (or iterator) provides an abstraction for a
> position within a ByteArray. A ByteArray only provides absolute access
> methods, and its Cursor provides only relative access methods. For
> consistency they should each have the same set of operations. A Cursor
> could also be Cloneable, so that users could emulate position/mark
> functionality if they want.
>
> In an implementation backed by a ByteBuffer, all absolute methods are
> fast, and for simplicity its Cursor just delegates to the absolute
> methods. In a composite implementation backed by a sequence of other
> ByteArrays, absolute methods are slow and relative methods are fast. And
> for simplicity, all absolute methods just create a cursor and call the
> equivalent relative method.
>
> Cheers
> Rich
>
> On Tue, Apr 29, 2008 at 5:43 PM, "이희승 (Trustin Lee)
> <trustin@gmail.com <ma...@gmail.com>> wrote:
>
> Hi Rich,
>
> Good to see you back in the mailing list. It seems like you were
> spending a fair amount of time with DIRMINA-489. :)
>
> Rich Dougherty wrote:
> <snip/>
> > - However, in a composite implementation, where the actual byte
> storage
> > is spread over a sequence of underlying objects, there may be a cost
> > associated with random access. Since a lot of real access is in fact
> > going to be sequential, it might make sense to provide something
> like an
> > "iterator" to efficiently support this usage pattern.
> >
> > In my (unfinished) CompositeIoBuffer implementation, I used a private
> > iterator class to implement the buffer's position and mark. This made
> > them a lot easier to use, since it encapsulated away the details
> of how
> > to efficiently traverse the component buffers.
>
> Could you share your iterator interface? I think it's worthwhile to
> review, too. I find myself start to like pastebin.com
> <http://pastebin.com> because they keep
> the code for a month. :)
>
> > - Does the ByteArray interface provide operations for expansion, or
> > would this be subclass-specific behaviour? I'm guessing expansion is a
> > subclass-specific, since the javadoc says that a ByteArray is
> fixed size.
>
> We could implement a special list of ByteArrays for expansion. For
> example, it could be something similar to Queue<ByteArray> with getter
> and putter methods.
>
> CompositeByteArray could remain fixed-size and be provided with a list
> of ByteArrays when it is constructed. WDYT?
>
> I initially thought we don't even need a CompositeByteArray, but I found
> it has its worth, especially with framing. A framing filter could
> assemble/frag an arbitrary number of ByteArrays into an arbitrary number
> of ByteArrays (i.e. N:M association). CompositeByteArray is essential
> to support this use case. For example, the current text line decoder
> could benefit from CompositeByteArray. Let's assume that the decoder
> received "AB" and "CD\nE". The decoder (or framing filter) could
> generate a CompositeByteArray of "AB" and PartialByteArray("CD\nE",
> 0, 2).
>
> > - Were you thinking that asByteBuffer() would return the ByteArray's
> > backing ByteBuffer, or on implementing a version of ByteBuffer that
> > delegates to the underlying ByteArray?
> >
> > If it's the former, then it's probably worth thinking about the
> desired
> > semantics for a CompositeByteArray. Do we need to guarantee that
> > modification of the returned array will always modify the underlying
> > ByteArray? I think it would be possible to /almost/ implement this
> for a
> > CompositeByteArray, i.e. by first copying all component buffers into a
> > single buffer, then returning its ByteBuffer. But if we then added
> > another buffer to the CompositeByteArray (expanding it) and called
> > asByteBuffer() again, we'd need to create another ByteBuffer. So we'd
> > now have two different ByteBuffers returned by this method, and the
> > first ByteBuffer would become "disconnected" from the underlying
> > ByteArray. I hope that made sense!
>
> You are right. If we are going to implement CompositeByteArray, we have
> to think a few things again:
>
> 1) we need to think again about asByteBuffer() method. asByteBuffer()
> method will be used as often as CompositeByteArray, and therefore
> copying on demand will kill the original intention of CompositeByteArray
> - zero-copy. It should return an array of ByteBuffer(s), or something
> different should be provided. So.. that's why I want to know how you
> implemented the iterator. :)
>
> 2) we need to add a minimal set of getter and setter methods such as
> getLong(index) and setInt(index, value), because offset calculation in
> CompositeByteArray is complicated so we can't just call get() or set()
> methods four to eight times - the overhead will not be negligible.
>
> What I can think of right now is something like this:
>
> ByteArray b = ...;
> ByteArrayPointer p = b.newPointer(8);
>
> assert p.index() == 8;
> // same with value = b.getInt(8); and p.index(p.index() + 4);
> long value = p.getInt();
> assert p.index() == 12;
>
> ByteArrayPointer is basically a wrapper of the ByteArray which provides
> stateful properties such as current index and byte order.
>
> So.. the introduction of CompositeByteArray makes direct exposure of NIO
> ByteBuffers somewhat difficult, although workarounds (disconnected
> buffer by on-demand/manual copy or returns an array of buffers) are
> available.
>
> WDYT?
>
> Thanks,
> --
> Trustin Lee - Principal Software Engineer, JBoss, Red Hat
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
>
>
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
Re: A quick prototyping of ByteArray interface
Posted by Rich Dougherty <ri...@rd.gen.nz>.
Yes, it has taken me a while to get that CompositeIoBuffer written. :-)
In any case it looks like the discussion has moved on past needing a
composite interface. But since I've already sketched something up I may as
well post it anyway!
http://pastebin.com/f3c9c5f8d
I've cut a lot of corners, but hopefully the idea is still clear.
In particular the Cursor (or iterator) provides an abstraction for a
position within a ByteArray. A ByteArray only provides absolute access
methods, and its Cursor provides only relative access methods. For
consistency they should each have the same set of operations. A Cursor could
also be Cloneable, so that users could emulate position/mark functionality
if they want.
In an implementation backed by a ByteBuffer, all absolute methods are fast,
and for simplicity its Cursor just delegates to the absolute methods. In a
composite implementation backed by a sequence of other ByteArrays, absolute
methods are slow and relative methods are fast. And for simplicity, all
absolute methods just create a cursor and call the equivalent relative
method.
Cheers
Rich
On Tue, Apr 29, 2008 at 5:43 PM, "이희승 (Trustin Lee) <tr...@gmail.com>
wrote:
> Hi Rich,
>
> Good to see you back in the mailing list. It seems like you were
> spending a fair amount of time with DIRMINA-489. :)
>
> Rich Dougherty wrote:
> <snip/>
> > - However, in a composite implementation, where the actual byte storage
> > is spread over a sequence of underlying objects, there may be a cost
> > associated with random access. Since a lot of real access is in fact
> > going to be sequential, it might make sense to provide something like an
> > "iterator" to efficiently support this usage pattern.
> >
> > In my (unfinished) CompositeIoBuffer implementation, I used a private
> > iterator class to implement the buffer's position and mark. This made
> > them a lot easier to use, since it encapsulated away the details of how
> > to efficiently traverse the component buffers.
>
> Could you share your iterator interface? I think it's worthwhile to
> review, too. I find myself start to like pastebin.com because they keep
> the code for a month. :)
>
> > - Does the ByteArray interface provide operations for expansion, or
> > would this be subclass-specific behaviour? I'm guessing expansion is a
> > subclass-specific, since the javadoc says that a ByteArray is fixed
> size.
>
> We could implement a special list of ByteArrays for expansion. For
> example, it could be something similar to Queue<ByteArray> with getter
> and putter methods.
>
> CompositeByteArray could remain fixed-size and be provided with a list
> of ByteArrays when it is constructed. WDYT?
>
> I initially thought we don't even need a CompositeByteArray, but I found
> it has its worth, especially with framing. A framing filter could
> assemble/frag an arbitrary number of ByteArrays into an arbitrary number
> of ByteArrays (i.e. N:M association). CompositeByteArray is essential
> to support this use case. For example, the current text line decoder
> could benefit from CompositeByteArray. Let's assume that the decoder
> received "AB" and "CD\nE". The decoder (or framing filter) could
> generate a CompositeByteArray of "AB" and PartialByteArray("CD\nE", 0, 2).
>
> > - Were you thinking that asByteBuffer() would return the ByteArray's
> > backing ByteBuffer, or on implementing a version of ByteBuffer that
> > delegates to the underlying ByteArray?
> >
> > If it's the former, then it's probably worth thinking about the desired
> > semantics for a CompositeByteArray. Do we need to guarantee that
> > modification of the returned array will always modify the underlying
> > ByteArray? I think it would be possible to /almost/ implement this for a
> > CompositeByteArray, i.e. by first copying all component buffers into a
> > single buffer, then returning its ByteBuffer. But if we then added
> > another buffer to the CompositeByteArray (expanding it) and called
> > asByteBuffer() again, we'd need to create another ByteBuffer. So we'd
> > now have two different ByteBuffers returned by this method, and the
> > first ByteBuffer would become "disconnected" from the underlying
> > ByteArray. I hope that made sense!
>
> You are right. If we are going to implement CompositeByteArray, we have
> to think a few things again:
>
> 1) we need to think again about asByteBuffer() method. asByteBuffer()
> method will be used as often as CompositeByteArray, and therefore
> copying on demand will kill the original intention of CompositeByteArray
> - zero-copy. It should return an array of ByteBuffer(s), or something
> different should be provided. So.. that's why I want to know how you
> implemented the iterator. :)
>
> 2) we need to add a minimal set of getter and setter methods such as
> getLong(index) and setInt(index, value), because offset calculation in
> CompositeByteArray is complicated so we can't just call get() or set()
> methods four to eight times - the overhead will not be negligible.
>
> What I can think of right now is something like this:
>
> ByteArray b = ...;
> ByteArrayPointer p = b.newPointer(8);
>
> assert p.index() == 8;
> // same with value = b.getInt(8); and p.index(p.index() + 4);
> long value = p.getInt();
> assert p.index() == 12;
>
> ByteArrayPointer is basically a wrapper of the ByteArray which provides
> stateful properties such as current index and byte order.
>
> So.. the introduction of CompositeByteArray makes direct exposure of NIO
> ByteBuffers somewhat difficult, although workarounds (disconnected
> buffer by on-demand/manual copy or returns an array of buffers) are
> available.
>
> WDYT?
>
> Thanks,
> --
> Trustin Lee - Principal Software Engineer, JBoss, Red Hat
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
>
>
Re: A quick prototyping of ByteArray interface
Posted by 이희승 (Trustin Lee) <t...@gmail.com>.
Here's the revised edition:
http://pastebin.com/m29a4dd44
"이희승 (Trustin Lee) <tr...@gmail.com>" wrote:
> Hi Rich,
>
> Good to see you back in the mailing list. It seems like you were
> spending a fair amount of time with DIRMINA-489. :)
>
> Rich Dougherty wrote:
> <snip/>
>> - However, in a composite implementation, where the actual byte storage
>> is spread over a sequence of underlying objects, there may be a cost
>> associated with random access. Since a lot of real access is in fact
>> going to be sequential, it might make sense to provide something like an
>> "iterator" to efficiently support this usage pattern.
>>
>> In my (unfinished) CompositeIoBuffer implementation, I used a private
>> iterator class to implement the buffer's position and mark. This made
>> them a lot easier to use, since it encapsulated away the details of how
>> to efficiently traverse the component buffers.
>
> Could you share your iterator interface? I think it's worthwhile to
> review, too. I find myself start to like pastebin.com because they keep
> the code for a month. :)
>
>> - Does the ByteArray interface provide operations for expansion, or
>> would this be subclass-specific behaviour? I'm guessing expansion is a
>> subclass-specific, since the javadoc says that a ByteArray is fixed size.
>
> We could implement a special list of ByteArrays for expansion. For
> example, it could be something similar to Queue<ByteArray> with getter
> and putter methods.
>
> CompositeByteArray could remain fixed-size and be provided with a list
> of ByteArrays when it is constructed. WDYT?
>
> I initially thought we don't even need a CompositeByteArray, but I found
> it has its worth, especially with framing. A framing filter could
> assemble/frag an arbitrary number of ByteArrays into an arbitrary number
> of ByteArrays (i.e. N:M association). CompositeByteArray is essential
> to support this use case. For example, the current text line decoder
> could benefit from CompositeByteArray. Let's assume that the decoder
> received "AB" and "CD\nE". The decoder (or framing filter) could
> generate a CompositeByteArray of "AB" and PartialByteArray("CD\nE", 0, 2).
>
>> - Were you thinking that asByteBuffer() would return the ByteArray's
>> backing ByteBuffer, or on implementing a version of ByteBuffer that
>> delegates to the underlying ByteArray?
>>
>> If it's the former, then it's probably worth thinking about the desired
>> semantics for a CompositeByteArray. Do we need to guarantee that
>> modification of the returned array will always modify the underlying
>> ByteArray? I think it would be possible to /almost/ implement this for a
>> CompositeByteArray, i.e. by first copying all component buffers into a
>> single buffer, then returning its ByteBuffer. But if we then added
>> another buffer to the CompositeByteArray (expanding it) and called
>> asByteBuffer() again, we'd need to create another ByteBuffer. So we'd
>> now have two different ByteBuffers returned by this method, and the
>> first ByteBuffer would become "disconnected" from the underlying
>> ByteArray. I hope that made sense!
>
> You are right. If we are going to implement CompositeByteArray, we have
> to think a few things again:
>
> 1) we need to think again about asByteBuffer() method. asByteBuffer()
> method will be used as often as CompositeByteArray, and therefore
> copying on demand will kill the original intention of CompositeByteArray
> - zero-copy. It should return an array of ByteBuffer(s), or something
> different should be provided. So.. that's why I want to know how you
> implemented the iterator. :)
>
> 2) we need to add a minimal set of getter and setter methods such as
> getLong(index) and setInt(index, value), because offset calculation in
> CompositeByteArray is complicated so we can't just call get() or set()
> methods four to eight times - the overhead will not be negligible.
>
> What I can think of right now is something like this:
>
> ByteArray b = ...;
> ByteArrayPointer p = b.newPointer(8);
>
> assert p.index() == 8;
> // same with value = b.getInt(8); and p.index(p.index() + 4);
> long value = p.getInt();
> assert p.index() == 12;
>
> ByteArrayPointer is basically a wrapper of the ByteArray which provides
> stateful properties such as current index and byte order.
>
> So.. the introduction of CompositeByteArray makes direct exposure of NIO
> ByteBuffers somewhat difficult, although workarounds (disconnected
> buffer by on-demand/manual copy or returns an array of buffers) are
> available.
>
> WDYT?
>
> Thanks,
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
Re: A quick prototyping of ByteArray interface
Posted by 이희승 (Trustin Lee) <t...@gmail.com>.
Hi Karl,
Outside - Karl's ACM wrote:
>> From: trustin@gmail.com [mailto:trustin@gmail.com]
>>
>> So.. the introduction of CompositeByteArray makes direct exposure of NIO
>> ByteBuffers somewhat difficult, although workarounds (disconnected
>> buffer by on-demand/manual copy or returns an array of buffers) are
>> available.
>>
>> WDYT?
>
> IMO the most essential use of multiple buffers is in the Scatter/Gather interfaces. So at least an array returning method is necessary.
>
> I'm also quite interested in the zero-copy front. Grizzly and MINA currently differ in this regard. I think that Grizzly without some modification has no support of zero-copy making it unsuitable for high volume data, for example something like a pure Java logging server with some kind of analysis/load to consume the data. I can begin testing this use case in MINA, but I have some ideas to make the memory utilization more manageable.
>
> In this use case some slices of buffers could persist for longer than others. One approach to minimize copies and allow more memory to be reclaimed would include evaluating allocating smaller buffers (perhaps 512 bytes or smaller, varied as part of the experiment) and persisting these until the data has been evaluated. These buffers could be periodically tested for their waste and copied to free up the waste. However, to keep reads efficient I would want to experiment with the GatheringByteChannel methods.
>
> For Grizzly I think this means creating a new ProtocolFilter and extending some part of the entire filter chain to support it. For MINA I will have to do a little more reading. Grizzly will also be relying entirely on the garbage collector to reclaim buffers and I may have to craft a class to track buffer splits, and by then I might as well implement a buffer pool.
Heap buffer pooling has been always slower than relying on garbage
collector from my experience. Direct buffer is somewhat different so it
requires more experimentation.
Thanks,
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
RE: A quick prototyping of ByteArray interface
Posted by Outside - Karl's ACM <kp...@acm.org>.
> From: trustin@gmail.com [mailto:trustin@gmail.com]
>
> So.. the introduction of CompositeByteArray makes direct exposure of NIO
> ByteBuffers somewhat difficult, although workarounds (disconnected
> buffer by on-demand/manual copy or returns an array of buffers) are
> available.
>
> WDYT?
IMO the most essential use of multiple buffers is in the Scatter/Gather interfaces. So at least an array returning method is necessary.
I'm also quite interested in the zero-copy front. Grizzly and MINA currently differ in this regard. I think that Grizzly without some modification has no support of zero-copy making it unsuitable for high volume data, for example something like a pure Java logging server with some kind of analysis/load to consume the data. I can begin testing this use case in MINA, but I have some ideas to make the memory utilization more manageable.
In this use case some slices of buffers could persist for longer than others. One approach to minimize copies and allow more memory to be reclaimed would include evaluating allocating smaller buffers (perhaps 512 bytes or smaller, varied as part of the experiment) and persisting these until the data has been evaluated. These buffers could be periodically tested for their waste and copied to free up the waste. However, to keep reads efficient I would want to experiment with the GatheringByteChannel methods.
For Grizzly I think this means creating a new ProtocolFilter and extending some part of the entire filter chain to support it. For MINA I will have to do a little more reading. Grizzly will also be relying entirely on the garbage collector to reclaim buffers and I may have to craft a class to track buffer splits, and by then I might as well implement a buffer pool.
-karl
Re: A quick prototyping of ByteArray interface
Posted by 이희승 (Trustin Lee) <t...@gmail.com>.
Hi Rich,
Good to see you back in the mailing list. It seems like you were
spending a fair amount of time with DIRMINA-489. :)
Rich Dougherty wrote:
<snip/>
> - However, in a composite implementation, where the actual byte storage
> is spread over a sequence of underlying objects, there may be a cost
> associated with random access. Since a lot of real access is in fact
> going to be sequential, it might make sense to provide something like an
> "iterator" to efficiently support this usage pattern.
>
> In my (unfinished) CompositeIoBuffer implementation, I used a private
> iterator class to implement the buffer's position and mark. This made
> them a lot easier to use, since it encapsulated away the details of how
> to efficiently traverse the component buffers.
Could you share your iterator interface? I think it's worthwhile to
review, too. I find myself start to like pastebin.com because they keep
the code for a month. :)
> - Does the ByteArray interface provide operations for expansion, or
> would this be subclass-specific behaviour? I'm guessing expansion is a
> subclass-specific, since the javadoc says that a ByteArray is fixed size.
We could implement a special list of ByteArrays for expansion. For
example, it could be something similar to Queue<ByteArray> with getter
and putter methods.
CompositeByteArray could remain fixed-size and be provided with a list
of ByteArrays when it is constructed. WDYT?
I initially thought we don't even need a CompositeByteArray, but I found
it has its worth, especially with framing. A framing filter could
assemble/frag an arbitrary number of ByteArrays into an arbitrary number
of ByteArrays (i.e. N:M association). CompositeByteArray is essential
to support this use case. For example, the current text line decoder
could benefit from CompositeByteArray. Let's assume that the decoder
received "AB" and "CD\nE". The decoder (or framing filter) could
generate a CompositeByteArray of "AB" and PartialByteArray("CD\nE", 0, 2).
> - Were you thinking that asByteBuffer() would return the ByteArray's
> backing ByteBuffer, or on implementing a version of ByteBuffer that
> delegates to the underlying ByteArray?
>
> If it's the former, then it's probably worth thinking about the desired
> semantics for a CompositeByteArray. Do we need to guarantee that
> modification of the returned array will always modify the underlying
> ByteArray? I think it would be possible to /almost/ implement this for a
> CompositeByteArray, i.e. by first copying all component buffers into a
> single buffer, then returning its ByteBuffer. But if we then added
> another buffer to the CompositeByteArray (expanding it) and called
> asByteBuffer() again, we'd need to create another ByteBuffer. So we'd
> now have two different ByteBuffers returned by this method, and the
> first ByteBuffer would become "disconnected" from the underlying
> ByteArray. I hope that made sense!
You are right. If we are going to implement CompositeByteArray, we have
to think a few things again:
1) we need to think again about asByteBuffer() method. asByteBuffer()
method will be used as often as CompositeByteArray, and therefore
copying on demand will kill the original intention of CompositeByteArray
- zero-copy. It should return an array of ByteBuffer(s), or something
different should be provided. So.. that's why I want to know how you
implemented the iterator. :)
2) we need to add a minimal set of getter and setter methods such as
getLong(index) and setInt(index, value), because offset calculation in
CompositeByteArray is complicated so we can't just call get() or set()
methods four to eight times - the overhead will not be negligible.
What I can think of right now is something like this:
ByteArray b = ...;
ByteArrayPointer p = b.newPointer(8);
assert p.index() == 8;
// same with value = b.getInt(8); and p.index(p.index() + 4);
long value = p.getInt();
assert p.index() == 12;
ByteArrayPointer is basically a wrapper of the ByteArray which provides
stateful properties such as current index and byte order.
So.. the introduction of CompositeByteArray makes direct exposure of NIO
ByteBuffers somewhat difficult, although workarounds (disconnected
buffer by on-demand/manual copy or returns an array of buffers) are
available.
WDYT?
Thanks,
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
Re: A quick prototyping of ByteArray interface
Posted by Rich Dougherty <ri...@rd.gen.nz>.
Hi Trustin
I've been looking into the CompositeIoBuffer issue (DIRMINA-489) over the
last few weeks, so I'm following the current discussion with interest.
Overall, I think your suggested interface is good, but I do have a few
comments and suggestions.
- I like the idea of providing an abstraction for a sequence of bytes that
has a relatively small set of operations. This will make it easier to
provide different implementations. (I was actually planning to write a code
generator to reduce the work involved writing all the different combinations
of absolute/relative getters/setters for each different type!)
- Removing the relative operations also makes it easier to share the
objects, since there is less state to be messed up.
- However, in a composite implementation, where the actual byte storage is
spread over a sequence of underlying objects, there may be a cost associated
with random access. Since a lot of real access is in fact going to be
sequential, it might make sense to provide something like an "iterator" to
efficiently support this usage pattern.
In my (unfinished) CompositeIoBuffer implementation, I used a private
iterator class to implement the buffer's position and mark. This made them a
lot easier to use, since it encapsulated away the details of how to
efficiently traverse the component buffers.
- Does the ByteArray interface provide operations for expansion, or would
this be subclass-specific behaviour? I'm guessing expansion is a
subclass-specific, since the javadoc says that a ByteArray is fixed size.
- Were you thinking that asByteBuffer() would return the ByteArray's backing
ByteBuffer, or on implementing a version of ByteBuffer that delegates to the
underlying ByteArray?
If it's the former, then it's probably worth thinking about the desired
semantics for a CompositeByteArray. Do we need to guarantee that
modification of the returned array will always modify the underlying
ByteArray? I think it would be possible to *almost* implement this for a
CompositeByteArray, i.e. by first copying all component buffers into a
single buffer, then returning its ByteBuffer. But if we then added another
buffer to the CompositeByteArray (expanding it) and called asByteBuffer()
again, we'd need to create another ByteBuffer. So we'd now have two
different ByteBuffers returned by this method, and the first ByteBuffer
would become "disconnected" from the underlying ByteArray. I hope that made
sense!
Cheers
Rich
On Tue, Apr 29, 2008 at 2:20 PM, "이희승 (Trustin Lee) <tr...@gmail.com>
wrote:
> Hi folks,
>
> I tried a quick prototyping of the ByteArray interface I've mentioned in
> our previous discussion thread. I wanted to check in the code right now
> for easier review, but ASF-wide SVN write access has been disabled due
> to some hardware issue. Let me paste it here:
>
> http://pastebin.com/m37fb44ba
>
> The idea is to implement a ByteBuffer-backed ByteArray class and
> implement some variable length buffer-of-arrays using it. WDYT?
>
> --
> Trustin Lee - Principal Software Engineer, JBoss, Red Hat
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
>
>
--
Rich Dougherty
http://www.richdougherty.com/
Re: A quick prototyping of ByteArray interface
Posted by 이희승 (Trustin Lee) <t...@gmail.com>.
Sure. As you see ByteArrayIterator in my prototype, we can simply add
order method there.
Thanks,
Steve Ulrich wrote:
> Hi!
>
> There's one thing more to point out: The byte order
> We're parsing messages from devices that are forwarded by a "gateway" to our servers. The gateway speaks big endian, while the device itselfs speaks little endian so we have to switch the byte order while parsing. The ByteBuffer solution is very nice for this, just a buf.order(ByteBuffer.LITTLE_ENDIAN); is enough.
>
> regards
>
> Steve
>
> Trustin Lee wrote:
>
>> -----Ursprüngliche Nachricht-----
>> Von: "이희승 (Trustin Lee) [mailto:trustin@gmail.com]
>> Gesendet: Dienstag, 29. April 2008 10:50
>> An: dev@mina.apache.org
>> Betreff: Re: A quick prototyping of ByteArray interface
>>
>> Julien Vermillard wrote:
>>> No get/set for other types than bytes ?
>> I think we should have get/set for all integer primitive types if we
>> are going to implement CompositeByteArray.
>>
>> For other types, we can choose one of the following:
>>
>> 1) Provide an AbstractByteArray which provides the default
>> implementation of all the utility getters and setters - this is what we
>> are doing right now.
>>
>> 2) Provide a utility class which is composed of many utility methods
>> and use static imports. For example:
>>
>> import static org.apache.mina.array.ArrayUtils.*;
>> String s = getPrefixedString(array)
>>
>> It seems like the second approach is gaining popularity recently in the
>> mailing list. I'd like to know what others think about this change
>> though.
>> --
>> Trustin Lee - Principal Software Engineer, JBoss, Red Hat
>> --
>> what we call human nature is actually human habit
>> --
>> http://gleamynode.net/
>
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
AW: A quick prototyping of ByteArray interface
Posted by Steve Ulrich <st...@proemion.com>.
Hi!
There's one thing more to point out: The byte order
We're parsing messages from devices that are forwarded by a "gateway" to our servers. The gateway speaks big endian, while the device itselfs speaks little endian so we have to switch the byte order while parsing. The ByteBuffer solution is very nice for this, just a buf.order(ByteBuffer.LITTLE_ENDIAN); is enough.
regards
Steve
Trustin Lee wrote:
> -----Ursprüngliche Nachricht-----
> Von: "이희승 (Trustin Lee) [mailto:trustin@gmail.com]
> Gesendet: Dienstag, 29. April 2008 10:50
> An: dev@mina.apache.org
> Betreff: Re: A quick prototyping of ByteArray interface
>
> Julien Vermillard wrote:
> > No get/set for other types than bytes ?
>
> I think we should have get/set for all integer primitive types if we
> are going to implement CompositeByteArray.
>
> For other types, we can choose one of the following:
>
> 1) Provide an AbstractByteArray which provides the default
> implementation of all the utility getters and setters - this is what we
> are doing right now.
>
> 2) Provide a utility class which is composed of many utility methods
> and use static imports. For example:
>
> import static org.apache.mina.array.ArrayUtils.*;
> String s = getPrefixedString(array)
>
> It seems like the second approach is gaining popularity recently in the
> mailing list. I'd like to know what others think about this change
> though.
> --
> Trustin Lee - Principal Software Engineer, JBoss, Red Hat
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
Re: A quick prototyping of ByteArray interface
Posted by 이희승 (Trustin Lee) <t...@gmail.com>.
Julien Vermillard wrote:
> On Tue, 29 Apr 2008 11:20:51 +0900 (KST)
> "이희승 (Trustin Lee) <tr...@gmail.com> wrote:
>
>> Hi folks,
>>
>> I tried a quick prototyping of the ByteArray interface I've mentioned
>> in our previous discussion thread. I wanted to check in the code
>> right now for easier review, but ASF-wide SVN write access has been
>> disabled due to some hardware issue. Let me paste it here:
>>
>> http://pastebin.com/m37fb44ba
>>
>> The idea is to implement a ByteBuffer-backed ByteArray class and
>> implement some variable length buffer-of-arrays using it. WDYT?
>>
>
> No get/set for other types than bytes ?
I think we should have get/set for all integer primitive types if we are
going to implement CompositeByteArray.
For other types, we can choose one of the following:
1) Provide an AbstractByteArray which provides the default
implementation of all the utility getters and setters - this is what we
are doing right now.
2) Provide a utility class which is composed of many utility methods and
use static imports. For example:
import static org.apache.mina.array.ArrayUtils.*;
String s = getPrefixedString(array)
It seems like the second approach is gaining popularity recently in the
mailing list. I'd like to know what others think about this change though.
--
Trustin Lee - Principal Software Engineer, JBoss, Red Hat
--
what we call human nature is actually human habit
--
http://gleamynode.net/
Re: A quick prototyping of ByteArray interface
Posted by Julien Vermillard <jv...@archean.fr>.
On Tue, 29 Apr 2008 11:20:51 +0900 (KST)
"이희승 (Trustin Lee) <tr...@gmail.com> wrote:
> Hi folks,
>
> I tried a quick prototyping of the ByteArray interface I've mentioned
> in our previous discussion thread. I wanted to check in the code
> right now for easier review, but ASF-wide SVN write access has been
> disabled due to some hardware issue. Let me paste it here:
>
> http://pastebin.com/m37fb44ba
>
> The idea is to implement a ByteBuffer-backed ByteArray class and
> implement some variable length buffer-of-arrays using it. WDYT?
>
No get/set for other types than bytes ?
Julien