You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@harmony.apache.org by "Vladimir Strigun (JIRA)" <ji...@apache.org> on 2006/04/25 16:39:05 UTC
[jira] Created: (HARMONY-410) method decode(ByteBuffer, CharBuffer,
boolean) should set correct position in ByteBuffer
method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
----------------------------------------------------------------------------------------
Key: HARMONY-410
URL: http://issues.apache.org/jira/browse/HARMONY-410
Project: Harmony
Type: Bug
Components: Classlib
Reporter: Vladimir Strigun
Priority: Minor
When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-410) method decode(ByteBuffer,
CharBuffer, boolean) should set correct position in ByteBuffer
Posted by "Richard Liang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=comments#action_12376394 ]
Richard Liang commented on HARMONY-410:
---------------------------------------
Hi, Vladimir
It's an expected behaviour on Harmony.
As spec says,
"CoderResult.UNDERFLOW indicates that as much of the input buffer as possible has been decoded. If there are no bytes remaining and the invoker has no further input then the decoding operation is complete. Otherwise there is insufficient input for the operation to proceed, so this method should be invoked again with further input. "
Therefore, Charset.decode saves remaining buffer for next invocation if invoker has further input. Since the remaining buffer has been stored in Charset.decode,
the position of "in" ByteBuffer is set to the limit, which means all the byte has been processed by decode.
In other words, Charset.decode takes the responsibility to store remaining bytes. It's streamming decode. Unfortunately, some RI doesn't implement as this.
Consider about following scenario:
decode(in1, out, false) then decode(in2, out, false)
If we depends on the remaining of "in" ByteBuffer, the code will look like:
decode(in1, out, false);
compoundIn2.put(in1);
compoundIn2.put(in2);
decode(compoundIn2);
I think that's not spec expected.
What's your opnion?
Thanks!
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Mikhail Loenko
> Priority: Minor
> Attachments: Harmony-410_patch.txt, harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Assigned: (HARMONY-410) method decode(ByteBuffer,
CharBuffer, boolean) should set correct position in ByteBuffer
Posted by "Mikhail Loenko (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=all ]
Mikhail Loenko reassigned HARMONY-410:
--------------------------------------
Assign To: Mikhail Loenko
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Mikhail Loenko
> Priority: Minor
> Attachments: Harmony-410_patch.txt, harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-410) method decode(ByteBuffer,
CharBuffer, boolean) should set correct position in ByteBuffer
Posted by "Vladimir Strigun (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=comments#action_12376273 ]
Vladimir Strigun commented on HARMONY-410:
------------------------------------------
The following test failed on Harmony:
public void testDecode_CharBuffer_ByteBuffer_boolean() {
byte[] arr = {-31, -96, -86, -32};
ByteBuffer bb = ByteBuffer.wrap(arr);
CharBuffer cb = CharBuffer.allocate(2);
CharsetDecoder decoder = Charset.forName("UTF-8").newDecoder();
decoder.decode(bb, cb, false);
assertTrue("remaining() in ByteBuffer is incorrect", bb.remaining() == 1);
}
I'm going to attach patch for tests.api.java.nio.charset.CharsetDecoderTest
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Priority: Minor
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (HARMONY-410) method decode(ByteBuffer, CharBuffer,
boolean) should set correct position in ByteBuffer
Posted by "Mikhail Loenko (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=all ]
Mikhail Loenko closed HARMONY-410:
----------------------------------
Resolution: Won't Fix
non-bug difference from RI
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Non-bug differences from RI
> Reporter: Vladimir Strigun
> Assignee: Mikhail Loenko
> Priority: Minor
> Attachments: Harmony-410_patch.txt, harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HARMONY-410) method decode(ByteBuffer, CharBuffer,
boolean) should set correct position in ByteBuffer
Posted by "Mikhail Loenko (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=all ]
Mikhail Loenko updated HARMONY-410:
-----------------------------------
Component: Non-bug differences from RI
(was: Classlib)
see
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200605.mbox/%3ce3b630230605150143v726c932ewc186d1fc6da7860@mail.gmail.com%3e
and
http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200605.mbox/%3c4d0b24970605220217m4b6c0de4r69d152054392b8db@mail.gmail.com%3e
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Non-bug differences from RI
> Reporter: Vladimir Strigun
> Assignee: Mikhail Loenko
> Priority: Minor
> Attachments: Harmony-410_patch.txt, harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-410) method decode(ByteBuffer,
CharBuffer, boolean) should set correct position in ByteBuffer
Posted by "Vladimir Strigun (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=comments#action_12376425 ]
Vladimir Strigun commented on HARMONY-410:
------------------------------------------
Hi Richard,
Thanks for the clarification, I agree that streaming decode is good thing, but I'd like to explain my understanding of specification :)
According to specification of CharsetDecoder decoding operation should process by the following steps:
"
2. Invoke the decode method zero or more times, as long as additional input may be available, passing false for the endOfInput argument and filling the input buffer and flushing the output buffer between invocations;
3. Invoke the decode method one final time, passing true for the endOfInput argument; and then
"
spec also says:
"The buffers' positions will be advanced to reflect the bytes read and the characters written, but their marks and limits will not be modified"
I understand these sentences in the next way:
invoke decode with endOfInput = false if you have additional input, then fill the buffer (i.e. add to buffer some additional input), invoke decode again and pass correct endOfInput parameter dependent of availability of input.
Example you provided is very useful and, of course, 1st option looks better, but what I'm suggest here is to reflect actual processed bytes in input. After first invocation of decode, not all bytes processed actually, i.e. almost all bytes processed, but some stored in decoder to further operation. Is it possible to store bytes in decoder, support streaming decoding, and, at the same time sets correct position in input buffer after each operation?
Thanks.
Vladimir.
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Mikhail Loenko
> Priority: Minor
> Attachments: Harmony-410_patch.txt, harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HARMONY-410) method decode(ByteBuffer, CharBuffer,
boolean) should set correct position in ByteBuffer
Posted by "Vladimir Strigun (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=all ]
Vladimir Strigun updated HARMONY-410:
-------------------------------------
Attachment: harmony-410_test.txt
regression test for the issue
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Priority: Minor
> Attachments: harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HARMONY-410) method decode(ByteBuffer, CharBuffer,
boolean) should set correct position in ByteBuffer
Posted by "Vladimir Strigun (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=all ]
Vladimir Strigun updated HARMONY-410:
-------------------------------------
Attachment: Harmony-410_patch.txt
Please try my patch.
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Priority: Minor
> Attachments: Harmony-410_patch.txt, harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-410) method decode(ByteBuffer,
CharBuffer, boolean) should set correct position in ByteBuffer
Posted by "Richard Liang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-410?page=comments#action_12376439 ]
Richard Liang commented on HARMONY-410:
---------------------------------------
Hi Vladimir,
I think we both understand the spec :)
Here I propose two solutions:
1. Set the ByteBuffer to the limit, and store the remaining ByteBuffer in decoder. Invokers doesn't care the position of in. If the result is UNDERFLOW and there're furthur input, just pass the new input to the decoder. It's a typical streaming decoder. That's what Harmony does currently.
2. Decoder doesn't store remaining ByteBuffer. Position of "in" is set to indicate the remaining ByteBuffer. Invoker should take care to generate new input ByteBuffer for next invocation. RI acts in this way.
Both are acceptable.
However, I think solution 1 is more reasonable.
"Is it possible to store bytes in decoder, support streaming decoding, and, at the same time sets correct position in input buffer after each operation? "
Yes. In fact, your patch will make Harmony act as the description above.
However, I don't think it solve the problem. Maybe invoker is more confusable and may think:
"Is the remaining bytebuffer maintained in decoder or in ? Shall I get the remaining buffer from in and pass it for next invocation?"
Anyway, we need a decision on this compatibility issue.
By the way, Vladimir, does solution one cause any problem on other classlib implementation?
Thanks !
> method decode(ByteBuffer, CharBuffer, boolean) should set correct position in ByteBuffer
> ----------------------------------------------------------------------------------------
>
> Key: HARMONY-410
> URL: http://issues.apache.org/jira/browse/HARMONY-410
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Mikhail Loenko
> Priority: Minor
> Attachments: Harmony-410_patch.txt, harmony-410_test.txt
>
> When ByteBuffer contain incomplete sequence of bytes for successful decoding, position in ByteBuffer should be set to latest successful byte. I will attach testcase and patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira