You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Santiago M. Mola (JIRA)" <ji...@apache.org> on 2014/09/24 11:44:35 UTC
[jira] [Commented] (FLUME-2215) ResettableFileInputStream can't
support ucs-4 character
[ https://issues.apache.org/jira/browse/FLUME-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146137#comment-14146137 ]
Santiago M. Mola commented on FLUME-2215:
-----------------------------------------
I'm using the second patch without any problem so far. In combination with decodeErrorPolicy=replace (or ignore) it seems to solve all Unicode parsing problems.
> ResettableFileInputStream can't support ucs-4 character
> --------------------------------------------------------
>
> Key: FLUME-2215
> URL: https://issues.apache.org/jira/browse/FLUME-2215
> Project: Flume
> Issue Type: Bug
> Affects Versions: v1.5.0
> Reporter: syntony liu
> Priority: Critical
> Labels: patch
> Attachments: FLUME-2215-0-README.txt, FLUME-2215-0.patch, FLUME-2215-1-README.txt, FLUME-2215-1.patch
>
>
> ResettableFileInputStream.java:readChar() not handle ucs-4 character. it need 2 charBuf. it cause an unexpected termination。
> a temporary solution:
> if (res.isOverflow() && !charBuf.hasRemaining()){
> logger.warn("decoder ucs-4 at postion: {}" , buf.position());
> tmpBuf.clear();
> res = decoder.decode(buf, tmpBuf, isEndOfInput);
> incrPosition( buf.position() - start, false);
> return '?';
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)