Hadoop Avro generated code error due to Turkish Locale

[Serkan Taş <se...@hotmail.com>]

> Hi all,
> 
> I am faced to a common problem of tr locale settings for java application while trying to build dev environment according tohttp://wiki.apache.org/hadoop/EclipseEnvironment.
> 
> Here is the error :
> 
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure
> [ERROR] /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244] unmappable character for encoding UTF-8
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244] unmappable character for encoding UTF-8
> 
> 
> I f i check the code i discovered the reason for error :
> 
>  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"AvroRecord\",\"namespace\":\"org.apache.hadoop.io.serializer.avro\",\"fields\":[{\"name\":\"intField\",\"type\":\"Ýnt\"}]}");
> 
> As you can see, locale dependent capitalization of letter "i" turns in to "Ý".
> 
> 



> This code is automatically generated by avro. 

Same bug exists in some other apache projects and fixed. I am not sure about avro

For eg.

OPENEJB-1071, OAK-260, IBATIS-218, etc.

Should i file a bug ?

Re: Hadoop Avro generated code error due to Turkish Locale

[Sean Busbey <bu...@cloudera.com>]

On Sat, Apr 5, 2014 at 11:49 AM, Serkan Taş <se...@hotmail.com> wrote:

> Hi all,
>
> I am faced to a common problem of tr locale settings for java application
> while trying to build dev environment according to
> http://wiki.apache.org/hadoop/EclipseEnvironment.
>
> Here is the error :
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
> (default-testCompile) on project hadoop-common: Compilation failure
> [ERROR]
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
> unmappable character for encoding UTF-8
> [ERROR] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile
> (default-testCompile) on project hadoop-common: Compilation failure
> /Users/serkan/programlar/dev/hadooptest/hadoop-trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/io/serializer/avro/AvroRecord.java:[10,244]
> unmappable character for encoding UTF-8
>
>
> I f i check the code i discovered the reason for error :
>
>  public static final org.apache.avro.Schema SCHEMA$ = new
> org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"AvroRecord\",\"namespace\":\"org.apache.hadoop.io.serializer.avro\",\"fields\":[{\"name\":\"intField\",\"type\":\"Ýnt\"}]}");
>
> As you can see, locale dependent capitalization of letter "i" turns in to
> "Ý".
>
>
>
>
> This code is automatically generated by avro.
>
>
> Same bug exists in some other apache projects and fixed. I am not sure
> about avro
>
> For eg.
>
> OPENEJB-1071 <https://issues.apache.org/jira/browse/OPENEJB-1071>, OAK-260<https://issues.apache.org/jira/browse/OAK-260>
> , IBATIS-218 <https://issues.apache.org/jira/browse/IBATIS-218>, etc.
>
> Should i file a bug ?
>
>
Yes, please do file a bug. Avro should be specifying a locale when
generating that Schema string.


-- 
Sean