You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/06/01 20:23:59 UTC
[jira] [Commented] (FLINK-3921) StringParser not specifying
encoding to use
[ https://issues.apache.org/jira/browse/FLINK-3921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311043#comment-15311043 ]
ASF GitHub Bot commented on FLINK-3921:
---------------------------------------
GitHub user rekhajoshm opened a pull request:
https://github.com/apache/flink/pull/2060
[FLINK-3921] StringParser encoding
Corrected StringParser encoding
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rekhajoshm/flink FLINK-3921
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2060.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2060
----
commit 1e1ce9efefccf5a5585be802af4200e9d1ae7a98
Author: Rekha Joshi <re...@gmail.com>
Date: 2016-06-01T20:03:50Z
Merge pull request #1 from apache/master
Apache Flink master pull
commit 675b6a44e76ae71901bc6d4eaea1d09b6f789ff6
Author: Joshi <re...@gmail.com>
Date: 2016-06-01T20:26:47Z
[FLINK-3921] StringParser encoding
----
> StringParser not specifying encoding to use
> -------------------------------------------
>
> Key: FLINK-3921
> URL: https://issues.apache.org/jira/browse/FLINK-3921
> Project: Flink
> Issue Type: Improvement
> Components: Core
> Affects Versions: 1.0.3
> Reporter: Tatu Saloranta
> Assignee: Rekha Joshi
> Priority: Trivial
>
> Class `flink.types.parser.StringParser` has javadocs indicating that contents are expected to be Ascii, similar to `StringValueParser`. That makes sense, but when constructing actual instance, no encoding is specified; on line 66 f.ex:
> this.result = new String(bytes, startPos+1, i - startPos - 2);
> which leads to using whatever default platform encoding is. If contents really are always Ascii (would not count on that as parser is used from CSV reader), not a big deal, but it can lead to the usual Latin-1-VS-UTF-8 issues.
> So I think that encoding should be explicitly specified, whatever is to be used: javadocs claim ascii, so could be "us-ascii", but could well be UTF-8 or even ISO-8859-1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)