You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2017/04/04 08:49:42 UTC

[jira] [Assigned] (SPARK-20156) Java String toLowerCase "Turkish locale bug" causes Spark problems

     [ https://issues.apache.org/jira/browse/SPARK-20156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen reassigned SPARK-20156:
---------------------------------

    Assignee: Sean Owen
     Summary: Java String toLowerCase "Turkish locale bug" causes Spark problems  (was: Local dependent library used for upper and lowercase conversions.)

I retitled this; please refer to things like http://mattryall.net/blog/2009/02/the-infamous-turkish-locale-bug for back-story on this particular issue. 

I believe the best change is to make all case-changing operations use Locale.ROOT.

> Java String toLowerCase "Turkish locale bug" causes Spark problems
> ------------------------------------------------------------------
>
>                 Key: SPARK-20156
>                 URL: https://issues.apache.org/jira/browse/SPARK-20156
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Shell
>    Affects Versions: 2.1.0
>         Environment: Ubunutu 16.04
> Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_121)
>            Reporter: Serkan Taş
>            Assignee: Sean Owen
>         Attachments: sprk_shell.txt
>
>
> If the regional setting of the operation system is Turkish, the famous java locale problem occurs (https://jira.atlassian.com/browse/CONF-5931 or https://issues.apache.org/jira/browse/AVRO-1493). 
> e.g : 
> "SERDEINFO" lowers to "serdeınfo"
> "uniquetable" uppers to "UNİQUETABLE"
> work around : 
> add -Duser.country=US -Duser.language=en to the end of the line 
> SPARK_SUBMIT_OPTS="$SPARK_SUBMIT_OPTS -Dscala.usejavacp=true"
> in spark-shell.sh



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org