[jira] Reopened: (XERCESJ-1257) buffer overflow in UTF8Reader for characters out of BMP

["Robert Stojnic (JIRA)" <xe...@xml.apache.org>]

     [ https://issues.apache.org/jira/browse/XERCESJ-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Stojnic reopened XERCESJ-1257:
-------------------------------------



I don't think it's a good practice to split surrogates between reads, since I think that most users and APIs expect a valid sequence of chars when they do a single buffered read. This seems to be the case with xerces as well, now I get the following exception:

org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 2 of 4-byte UTF-8 sequence.
        at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
        at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
        at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
        at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
        at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
        .....

You're right, my patch shouldn't be comparing out+1 with ch.length, one should probably do something like:
if (out + 1 >= offset + length) {
...
}

I would attach the original XML I use, but it's 20GB, and the problematic part is somewhere in the middle :)



> buffer overflow in UTF8Reader for characters out of BMP
> -------------------------------------------------------
>
>                 Key: XERCESJ-1257
>                 URL: https://issues.apache.org/jira/browse/XERCESJ-1257
>             Project: Xerces2-J
>          Issue Type: Bug
>          Components: JAXP (javax.xml.parsers)
>    Affects Versions: 2.9.0
>         Environment: Any
>            Reporter: Robert Stojnic
>            Assignee: Michael Glavassevich
>            Priority: Minor
>         Attachments: TestXerces.java, UTF8Reader.patch
>
>
> There is a ArrayOutOfBoundsException in org.apache.xerces.impl.io.UTF8Reader, in read(char[],int,int) for 4-byte utf-8 chars.
> Imagine a following scenario. read() has a buffer of size N, and it reads N-1 ascii chars, and stores it in the output buffer. Let the Nth char be the first byte of a 4 byte utf-8 char. The other 3 bytes are fetched by invoking read() on the input stream. From these a surrogate pair of java chars is made, however, method does not check if both chars can fit into the output buffer ... In most cases, they would fit into the ouput buffer (e.g. if there are some other multi-byte chars in the fetched text), so the bug is very rare, but it still happens.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: j-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-dev-help@xerces.apache.org