You are viewing a plain text version of this content. The canonical link for it is here.
Posted to c-dev@xerces.apache.org by "Simon White (JIRA)" <xe...@xml.apache.org> on 2012/06/07 11:37:23 UTC
[jira] [Created] (XERCESC-1987) Transcoding Issue with single XMLCh
to utf8
Simon White created XERCESC-1987:
------------------------------------
Summary: Transcoding Issue with single XMLCh to utf8
Key: XERCESC-1987
URL: https://issues.apache.org/jira/browse/XERCESC-1987
Project: Xerces-C++
Issue Type: Bug
Components: Utilities
Affects Versions: 2.8.0
Environment: Windows XP
Reporter: Simon White
There appears to be an issue with transcoding to utf8. Conditions:
Input string = Single Chinese Character (XmlCh holds value 27493).
Problem code in TranscodeToStr::transcode:
unsigned int allocSize = len * sizeof(XMLCh);
fString = (XMLByte*)fMemoryManager->allocate(allocSize);
This code sets the output buffer to be two bytes. The issue here is that the character in question converts to a 3 byte utf8 character. It therefore hits this in XMLUTF8Transcoder.cpp:
// If we cannot fully get this char into the output buffer,
// then leave it for the next time.
//
if (outPtr + encodedBytes > outEnd)
break;
Since this is only a single character being converted it returns 0 and then hits this since nothing could be decoded:
if(charsRead == 0)
ThrowXMLwithMemMgr(TranscodingException, XMLExcepts::Trans_BadSrcSeq, fMemoryManager);
The sequence is not invalid, only output buffer has been limited to input buffer size. Is simply adding a few spare characters to allocSize the correct fix?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org
[jira] [Commented] (XERCESC-1987) Transcoding Issue with single
XMLCh to utf8
Posted by "Lee Doron (JIRA)" <xe...@xml.apache.org>.
[ https://issues.apache.org/jira/browse/XERCESC-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13293150#comment-13293150 ]
Lee Doron commented on XERCESC-1987:
------------------------------------
There is a partial fix for this kind of issue, for v3.1.1, in XERCESC-1947 (see Subversion Commit #1138012); the subsequent part is in the comments to XERCESC-1984. I haven't worked with v2.8, but I imagine you can probably backport the fix. If you do, please attach your patches here for the benefit of other users.
> Transcoding Issue with single XMLCh to utf8
> -------------------------------------------
>
> Key: XERCESC-1987
> URL: https://issues.apache.org/jira/browse/XERCESC-1987
> Project: Xerces-C++
> Issue Type: Bug
> Components: Utilities
> Affects Versions: 2.8.0
> Environment: Windows XP
> Reporter: Simon White
>
> There appears to be an issue with transcoding to utf8. Conditions:
> Input string = Single Chinese Character (XmlCh holds value 27493).
> Problem code in TranscodeToStr::transcode:
> unsigned int allocSize = len * sizeof(XMLCh);
> fString = (XMLByte*)fMemoryManager->allocate(allocSize);
> This code sets the output buffer to be two bytes. The issue here is that the character in question converts to a 3 byte utf8 character. It therefore hits this in XMLUTF8Transcoder.cpp:
> // If we cannot fully get this char into the output buffer,
> // then leave it for the next time.
> //
> if (outPtr + encodedBytes > outEnd)
> break;
> Since this is only a single character being converted it returns 0 and then hits this since nothing could be decoded:
> if(charsRead == 0)
> ThrowXMLwithMemMgr(TranscodingException, XMLExcepts::Trans_BadSrcSeq, fMemoryManager);
> The sequence is not invalid, only output buffer has been limited to input buffer size. Is simply adding a few spare characters to allocSize the correct fix?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: c-dev-unsubscribe@xerces.apache.org
For additional commands, e-mail: c-dev-help@xerces.apache.org