You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by "Vladimir Strigun (JIRA)" <ji...@apache.org> on 2006/01/17 14:59:42 UTC
[jira] Created: (HARMONY-33) java.nio.BufferOverflow exception
while decoding ByteBuffer with UTF-16 charset
java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
-------------------------------------------------------------------------------
Key: HARMONY-33
URL: http://issues.apache.org/jira/browse/HARMONY-33
Project: Harmony
Type: Bug
Components: Classlib
Reporter: Vladimir Strigun
Assigned to: Geir Magnusson Jr
Priority: Minor
If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
import java.nio.*;
import java.nio.charset.*;
import junit.framework.TestCase;
public class TestDecoder extends TestCase {
public static void main(String[] args) {
junit.textui.TestRunner.run(TestDecoder.class);
}
public static void testDecoder(){
try{
ByteBuffer bb = ByteBuffer.allocate(1);
bb.put(0,(byte)77);
CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
}catch(Exception e){
fail("Exception occured: "+e);
}
}
}
Output on RI:
.
Time: 0,03
OK (1 test)
Output with Harmony:
.F
Time: 0,01
There was 1 failure:
1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
at TestDecoder.testDecoder(TestDecoder.java:20)
at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
at TestDecoder.main(TestDecoder.java:10)
FAILURES!!!
Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-33) java.nio.BufferOverflow exception
while decoding ByteBuffer with UTF-16 charset
Posted by "Vladimir Strigun (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-33?page=comments#action_12363090 ]
Vladimir Strigun commented on HARMONY-33:
-----------------------------------------
Hi Paulex,
I can't reproduce IllegalStateException with my fix and with next test:
import java.nio.*;
import java.nio.charset.*;
public class Harmony33 {
public static void main(String[] args) {
try{
ByteBuffer bb = ByteBuffer.allocate(1);
bb.put(0,(byte)77);
CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
}catch(Exception e){
e.printStackTrace();
}
}
}
Possibly, you used not complete fix. My initial fix remove line "continue;" after checking remaining value of CharBuffer. But if I don't remove "continue;" line after the check I get same exception:
java.lang.IllegalStateException
at java.nio.charset.CharsetDecoder.flush(Unknown Source)
at java.nio.charset.CharsetDecoder.decode(Unknown Source)
at Harmony33.main(Harmony33.java:11)
> java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
> -------------------------------------------------------------------------------
>
> Key: HARMONY-33
> URL: http://issues.apache.org/jira/browse/HARMONY-33
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Geir Magnusson Jr
> Priority: Minor
>
> If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
> import java.nio.*;
> import java.nio.charset.*;
> import junit.framework.TestCase;
> public class TestDecoder extends TestCase {
> public static void main(String[] args) {
> junit.textui.TestRunner.run(TestDecoder.class);
> }
> public static void testDecoder(){
> try{
> ByteBuffer bb = ByteBuffer.allocate(1);
> bb.put(0,(byte)77);
> CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
> CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
> }catch(Exception e){
> fail("Exception occured: "+e);
> }
> }
> }
> Output on RI:
> .
> Time: 0,03
> OK (1 test)
> Output with Harmony:
> .F
> Time: 0,01
> There was 1 failure:
> 1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
> at TestDecoder.testDecoder(TestDecoder.java:20)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
> at TestDecoder.main(TestDecoder.java:10)
> FAILURES!!!
> Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Assigned: (HARMONY-33) java.nio.BufferOverflow exception
while decoding ByteBuffer with UTF-16 charset
Posted by "Tim Ellison (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-33?page=all ]
Tim Ellison reassigned HARMONY-33:
----------------------------------
Assign To: Tim Ellison (was: Geir Magnusson Jr)
> java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
> -------------------------------------------------------------------------------
>
> Key: HARMONY-33
> URL: http://issues.apache.org/jira/browse/HARMONY-33
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Tim Ellison
> Priority: Minor
>
> If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
> import java.nio.*;
> import java.nio.charset.*;
> import junit.framework.TestCase;
> public class TestDecoder extends TestCase {
> public static void main(String[] args) {
> junit.textui.TestRunner.run(TestDecoder.class);
> }
> public static void testDecoder(){
> try{
> ByteBuffer bb = ByteBuffer.allocate(1);
> bb.put(0,(byte)77);
> CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
> CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
> }catch(Exception e){
> fail("Exception occured: "+e);
> }
> }
> }
> Output on RI:
> .
> Time: 0,03
> OK (1 test)
> Output with Harmony:
> .F
> Time: 0,01
> There was 1 failure:
> 1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
> at TestDecoder.testDecoder(TestDecoder.java:20)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
> at TestDecoder.main(TestDecoder.java:10)
> FAILURES!!!
> Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-33) java.nio.BufferOverflow exception
while decoding ByteBuffer with UTF-16 charset
Posted by "Vladimir Strigun (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-33?page=comments#action_12363005 ]
Vladimir Strigun commented on HARMONY-33:
-----------------------------------------
The reason of the issue is invocation of put(replacement) method for CharBuffer with zero capacity. First, during decoding byte buffer, char buffer should be created with initial capacity equals to remaining bytes in byte buffer multiplied by average chars per byte value. For the current testcase initial capacity for char buffer is 0 ( int length = (int) (in.remaining() * averChars) ). And when CodingErrorAction = CodingErrorAction.REPLACE decoder try to put replacement string to zero-sized char buffer.
Please review fix for the issue (diff for revision 367369 of /incubator/harmony/enhanced/classlib/trunk/modules/nio_char/src/main/java/java/nio/charset/CharsetDecoder.java):
405a406,408
> if(out.remaining() < replace.length() ) {
> return CoderResult.OVERFLOW;
> }
407d409
< continue;
> java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
> -------------------------------------------------------------------------------
>
> Key: HARMONY-33
> URL: http://issues.apache.org/jira/browse/HARMONY-33
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Geir Magnusson Jr
> Priority: Minor
>
> If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
> import java.nio.*;
> import java.nio.charset.*;
> import junit.framework.TestCase;
> public class TestDecoder extends TestCase {
> public static void main(String[] args) {
> junit.textui.TestRunner.run(TestDecoder.class);
> }
> public static void testDecoder(){
> try{
> ByteBuffer bb = ByteBuffer.allocate(1);
> bb.put(0,(byte)77);
> CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
> CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
> }catch(Exception e){
> fail("Exception occured: "+e);
> }
> }
> }
> Output on RI:
> .
> Time: 0,03
> OK (1 test)
> Output with Harmony:
> .F
> Time: 0,01
> There was 1 failure:
> 1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
> at TestDecoder.testDecoder(TestDecoder.java:20)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
> at TestDecoder.main(TestDecoder.java:10)
> FAILURES!!!
> Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (HARMONY-33) java.nio.BufferOverflow exception while
decoding ByteBuffer with UTF-16 charset
Posted by "Tim Ellison (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-33?page=all ]
Tim Ellison closed HARMONY-33:
------------------------------
Validated by Vladimir.
> java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
> -------------------------------------------------------------------------------
>
> Key: HARMONY-33
> URL: http://issues.apache.org/jira/browse/HARMONY-33
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Tim Ellison
> Priority: Minor
>
> If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
> import java.nio.*;
> import java.nio.charset.*;
> import junit.framework.TestCase;
> public class TestDecoder extends TestCase {
> public static void main(String[] args) {
> junit.textui.TestRunner.run(TestDecoder.class);
> }
> public static void testDecoder(){
> try{
> ByteBuffer bb = ByteBuffer.allocate(1);
> bb.put(0,(byte)77);
> CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
> CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
> }catch(Exception e){
> fail("Exception occured: "+e);
> }
> }
> }
> Output on RI:
> .
> Time: 0,03
> OK (1 test)
> Output with Harmony:
> .F
> Time: 0,01
> There was 1 failure:
> 1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
> at TestDecoder.testDecoder(TestDecoder.java:20)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
> at TestDecoder.main(TestDecoder.java:10)
> FAILURES!!!
> Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-33) java.nio.BufferOverflow exception
while decoding ByteBuffer with UTF-16 charset
Posted by "Paulex Yang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-33?page=comments#action_12363077 ]
Paulex Yang commented on HARMONY-33:
------------------------------------
I agree with the cause, and the fix seems reasonable, but when I tried to apply the fix , and ran the test provided, it throws another exception like this:
java.lang.IllegalStateException
at java.nio.charset.CharsetDecoder.flush(CharsetDecoder.java:516)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:244)
at bugtests.Harmony33.main(Harmony33.java:15)
Looking further into the CharsetDecoder.decode(ByteBuffer), I found the decode(ByteBuffer) implements a whole decode procedure, which includes several steps, just as the JavaDoc said(I copied it here):
<quote>
a decoding operation:
* Invoking the reset() method to reset the decoder if the decoder has been used;
* Invoking the decode(ByteBuffer, CharBuffer, boolean) method until the addtional input is not needed, the code>endOfInput parameter must be set to false, the input buffer must be filled and the output buffer must be flushed between invocations;
* Invoking the decode(ByteBuffer, CharBuffer, boolean) method last time, and the the endOfInput parameter must be set to true
* Invoking the flush(CharBuffer) method to flush the output.
</quote>
So it will end the decode procedure by flush(CharBuffer). And the flush() method will throw java.lang.IllegalStateException unless the CharsetDecoder's is in END or INIT status, the relevant code is like this:
<quote>
public final CoderResult flush(CharBuffer out) {
if (status != END && status != INIT) {
throw new IllegalStateException();
}
..
}
</quote>
Now let's look back the original fix again, it return the CoderResult.OVERFLOW directly without change the CharDecoder's status so that the following flush() method will throw exception. A better alternatives to this fix is:
....
if(out.remaining() < replace.length() ) {
result = CoderResult.OVERFLOW;
} else{
out.put(replace);
continue;
}
.....
So that it will then break out of the while loop, set the status, and return. Test shows it will exit normally.
P.S. I'm still suspicous about the alternative fix in fact, because in some case the replace character isn't really insert into the output CharBuffer, I'm not sure if it is correct behaviour and I will study it with more tests later:).
> java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
> -------------------------------------------------------------------------------
>
> Key: HARMONY-33
> URL: http://issues.apache.org/jira/browse/HARMONY-33
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Geir Magnusson Jr
> Priority: Minor
>
> If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
> import java.nio.*;
> import java.nio.charset.*;
> import junit.framework.TestCase;
> public class TestDecoder extends TestCase {
> public static void main(String[] args) {
> junit.textui.TestRunner.run(TestDecoder.class);
> }
> public static void testDecoder(){
> try{
> ByteBuffer bb = ByteBuffer.allocate(1);
> bb.put(0,(byte)77);
> CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
> CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
> }catch(Exception e){
> fail("Exception occured: "+e);
> }
> }
> }
> Output on RI:
> .
> Time: 0,03
> OK (1 test)
> Output with Harmony:
> .F
> Time: 0,01
> There was 1 failure:
> 1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
> at TestDecoder.testDecoder(TestDecoder.java:20)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
> at TestDecoder.main(TestDecoder.java:10)
> FAILURES!!!
> Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (HARMONY-33) java.nio.BufferOverflow exception
while decoding ByteBuffer with UTF-16 charset
Posted by "Tim Ellison (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-33?page=all ]
Tim Ellison resolved HARMONY-33:
--------------------------------
Resolution: Fixed
Vladimir,
Fixed in NIO_CHAR module java.nio.charset.CharsetDecoder at repo revision 377729.
Please check that this fully resolves your problem.
> java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
> -------------------------------------------------------------------------------
>
> Key: HARMONY-33
> URL: http://issues.apache.org/jira/browse/HARMONY-33
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Tim Ellison
> Priority: Minor
>
> If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
> import java.nio.*;
> import java.nio.charset.*;
> import junit.framework.TestCase;
> public class TestDecoder extends TestCase {
> public static void main(String[] args) {
> junit.textui.TestRunner.run(TestDecoder.class);
> }
> public static void testDecoder(){
> try{
> ByteBuffer bb = ByteBuffer.allocate(1);
> bb.put(0,(byte)77);
> CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
> CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
> }catch(Exception e){
> fail("Exception occured: "+e);
> }
> }
> }
> Output on RI:
> .
> Time: 0,03
> OK (1 test)
> Output with Harmony:
> .F
> Time: 0,01
> There was 1 failure:
> 1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
> at TestDecoder.testDecoder(TestDecoder.java:20)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
> at TestDecoder.main(TestDecoder.java:10)
> FAILURES!!!
> Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-33) java.nio.BufferOverflow exception
while decoding ByteBuffer with UTF-16 charset
Posted by "Vladimir Strigun (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-33?page=comments#action_12366344 ]
Vladimir Strigun commented on HARMONY-33:
-----------------------------------------
Tim, thank you, I can't reproduce it with latest sources.
> java.nio.BufferOverflow exception while decoding ByteBuffer with UTF-16 charset
> -------------------------------------------------------------------------------
>
> Key: HARMONY-33
> URL: http://issues.apache.org/jira/browse/HARMONY-33
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Vladimir Strigun
> Assignee: Tim Ellison
> Priority: Minor
>
> If I try to decode ByteBuffer with lengh 1 using UTF-16 decoder unexpected java.nio.BufferOverflow exception occured. Please see testcase below.
> import java.nio.*;
> import java.nio.charset.*;
> import junit.framework.TestCase;
> public class TestDecoder extends TestCase {
> public static void main(String[] args) {
> junit.textui.TestRunner.run(TestDecoder.class);
> }
> public static void testDecoder(){
> try{
> ByteBuffer bb = ByteBuffer.allocate(1);
> bb.put(0,(byte)77);
> CharsetDecoder utf16D = Charset.forName("UTF-16").newDecoder();
> CharBuffer cb = utf16D.onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
> }catch(Exception e){
> fail("Exception occured: "+e);
> }
> }
> }
> Output on RI:
> .
> Time: 0,03
> OK (1 test)
> Output with Harmony:
> .F
> Time: 0,01
> There was 1 failure:
> 1) testDecoder(TestDecoder)junit.framework.AssertionFailedError: Exception occured: java.nio.BufferOverflowException
> at TestDecoder.testDecoder(TestDecoder.java:20)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:211)
> at TestDecoder.main(TestDecoder.java:10)
> FAILURES!!!
> Tests run: 1, Failures: 1, Errors: 0
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira