You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Tejas Patil (JIRA)" <ji...@apache.org> on 2016/06/08 18:48:20 UTC

[jira] [Updated] (SPARK-15826) PipedRDD to strictly use UTF-8 and not rely on default encoding

     [ https://issues.apache.org/jira/browse/SPARK-15826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tejas Patil updated SPARK-15826:
--------------------------------
    Summary: PipedRDD to strictly use UTF-8 and not rely on default encoding  (was: PipedRDD relies on JVM default encoding)

> PipedRDD to strictly use UTF-8 and not rely on default encoding
> ---------------------------------------------------------------
>
>                 Key: SPARK-15826
>                 URL: https://issues.apache.org/jira/browse/SPARK-15826
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>            Reporter: Tejas Patil
>            Priority: Trivial
>
> Encountered an issue wherein the code works in some cluster but fails on another one for the same input. After debugging realised that PipedRDD is picking default char encoding from the JVM which may be different across different platforms. 
> Making it use UTF-8 encoding just like `ScriptTransformation` does



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org