[jira] [Commented] (AVRO-1810) GenericDatumWriter broken with Enum

["Reuben Kuhnert (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15504015#comment-15504015 ] 

Reuben Kuhnert commented on AVRO-1810:
--------------------------------------

Sup Sean, getting burned by this:

{code}
org.apache.avro.AvroTypeException: Not an enum: UNIT
	at org.apache.avro.generic.GenericDatumWriter.writeEnum(GenericDatumWriter.java:164)
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:106)
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
	at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
{code}

Looks like enums have to subclass [GenericEnumSymbol|https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java#L800]. Fair enough. But, it looks like the class auto-generation tools that we're using are not putting that into the schema. We're using:

{code}
                 <plugin>
  <groupId>org.apache.avro</groupId>
  <artifactId>avro-maven-plugin</artifactId>
  <version>${avro.version}</version>
  <executions>
    <execution>
      <phase>generate-sources</phase>
      <goals>
        <goal>schema</goal>
        <goal>protocol</goal>
        <goal>idl-protocol</goal>
      </goals>
      <configuration>
        <fieldVisibility>PRIVATE</fieldVisibility>
      </configuration>
    </execution>
  </executions>
</plugin>
{code}

But it generates:

{code}
@org.apache.avro.specific.AvroGenerated
public enum EntityType { // LOOK MA, NO INTERFACE
  LISTING, UNIT  ;
  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"enum\",\"name\":\"EntityType\",\"namespace\":\"com.homeaway.commons.logging.events.lm\",\"symbols\":[\"LISTING\",\"UNIT\"]}");
  public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
}
{code}

For now, rolling back to {{1.7.7}}, but that does seem brokenish.


> GenericDatumWriter broken with Enum
> -----------------------------------
>
>                 Key: AVRO-1810
>                 URL: https://issues.apache.org/jira/browse/AVRO-1810
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.8.0
>            Reporter: Ryon Day
>            Priority: Blocker
>
> {panel:title=Description|titleBGColor=#3FA|bgColor=#DDD}
> Using the GenericDatumWriter with either Generic OR SpecificRecord will break if an Enum is present.
> {panel}
> {panel:title=Steps To Reproduce|titleBGColor=#8DB|bgColor=#DDD}
> I have been tracking Avro decoding oddities for a while.
> The tests for this issue can be found [here|https://github.com/ryonday/avroDecodingHelp/blob/master/src/test/java/com/ryonday/test/Avro180EnumFail.java]
> {panel}
> {panel:title=Notes|titleBGColor=#3AF|bgColor=#DDD}
> Due to the debacle that is the Avro "UTF8" object, we have been avoiding it by using the following scheme:
> * Write incoming records to a byte array using the GenericDatumWriter
> * Read back the byte array to our compiled Java domain objects using a SpecificDatumWriter
> This worked great with Avro 1.7.7, and this is a binary-incompatable breaking change with 1.8.0.
> This would appear to be caused by an addition in the {{GenericDatumWriter:163-164}}:
> {code}
>   if (!data.isEnum(datum))
>       throw new AvroTypeException("Not an enum: "+datum);
> {code}
> {panel}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)