You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Wojciech Sroka (JIRA)" <ji...@apache.org> on 2018/02/22 07:44:01 UTC
[jira] [Updated] (AVRO-2149) Using GenericDatumWriter for writing
GenericRecord with schema from the specified generated type
[ https://issues.apache.org/jira/browse/AVRO-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wojciech Sroka updated AVRO-2149:
---------------------------------
Description:
Currently using GenericDatumWriter for writing GenericRecord with schema from the specified generated type cause exception when one of fields is enum type:
{code:java}
Exception in thread "main" org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroTypeException: Not an enum: YELLOW
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at pl.ingbank.kafka.AvroErrorTester.runGenericWriterReaderTest(AvroErrorTester.java:65)
at pl.ingbank.kafka.AvroErrorTester.main(AvroErrorTester.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: org.apache.avro.AvroTypeException: Not an enum: YELLOW
at org.apache.avro.generic.GenericDatumWriter.writeEnum(GenericDatumWriter.java:195)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:128)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:177)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:167)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 7 more{code}
It is caused by method protected void writeEnum(Schema schema, Object datum, Encoder out) in class org.apache.avro.generic.GenericDatumWriter:
{code:java}
protected void writeEnum(Schema schema, Object datum, Encoder out) throws IOException {
if (!data.isEnum(datum)) {
throw new AvroTypeException("Not an enum: " + datum);
}
out.writeEnum(schema.getEnumOrdinal(datum.toString()));
}{code}
Fix for this is to add additional check does datum is enum type:
{code:java}
protected void writeEnum(Schema schema, Object datum, Encoder out) throws IOException {
if(datum.getClass().isEnum() && !data.isEnum(datum)){
throw new AvroTypeException("Not an enum: " + datum);
}
out.writeEnum(schema.getEnumOrdinal(datum.toString()));
}
{code}
This fix alows to serialize and deserialize generic record with schema from the specified generated type.
was:
Currently using GenericDatumWriter for writing GenericRecord with schema from the specified generated type cause exception when one of fields is enum type:
{code:java}
Exception in thread "main" org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroTypeException: Not an enum: YELLOW
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at pl.ingbank.kafka.AvroErrorTester.runGenericWriterReaderTest(AvroErrorTester.java:65)
at pl.ingbank.kafka.AvroErrorTester.main(AvroErrorTester.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: org.apache.avro.AvroTypeException: Not an enum: YELLOW
at org.apache.avro.generic.GenericDatumWriter.writeEnum(GenericDatumWriter.java:195)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:128)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:177)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:167)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 7 more{code}
It is caused by method protected void writeEnum(Schema schema, Object datum, Encoder out) in class org.apache.avro.generic.GenericDatumWriter:
{code:java}
if (!data.isEnum(datum)) {
throw new AvroTypeException("Not an enum: " + datum);
}{code}
Fix for this is to add additional check does datum is enum type:
{code:java}
if(datum.getClass().isEnum() && !data.isEnum(datum)){
throw new AvroTypeException("Not an enum: " + datum);
}
{code}
This fix alows to serialize and deserialize generic record with schema from the specified generated type.
> Using GenericDatumWriter for writing GenericRecord with schema from the specified generated type
> ------------------------------------------------------------------------------------------------
>
> Key: AVRO-2149
> URL: https://issues.apache.org/jira/browse/AVRO-2149
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.8.2
> Reporter: Wojciech Sroka
> Priority: Major
>
> Currently using GenericDatumWriter for writing GenericRecord with schema from the specified generated type cause exception when one of fields is enum type:
> {code:java}
> Exception in thread "main" org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroTypeException: Not an enum: YELLOW
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at pl.ingbank.kafka.AvroErrorTester.runGenericWriterReaderTest(AvroErrorTester.java:65)
> at pl.ingbank.kafka.AvroErrorTester.main(AvroErrorTester.java:27)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Caused by: org.apache.avro.AvroTypeException: Not an enum: YELLOW
> at org.apache.avro.generic.GenericDatumWriter.writeEnum(GenericDatumWriter.java:195)
> at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:128)
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
> at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:177)
> at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:167)
> at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118)
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
> at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> ... 7 more{code}
> It is caused by method protected void writeEnum(Schema schema, Object datum, Encoder out) in class org.apache.avro.generic.GenericDatumWriter:
> {code:java}
> protected void writeEnum(Schema schema, Object datum, Encoder out) throws IOException {
> if (!data.isEnum(datum)) {
> throw new AvroTypeException("Not an enum: " + datum);
> }
> out.writeEnum(schema.getEnumOrdinal(datum.toString()));
> }{code}
> Fix for this is to add additional check does datum is enum type:
> {code:java}
> protected void writeEnum(Schema schema, Object datum, Encoder out) throws IOException {
> if(datum.getClass().isEnum() && !data.isEnum(datum)){
> throw new AvroTypeException("Not an enum: " + datum);
> }
> out.writeEnum(schema.getEnumOrdinal(datum.toString()));
> }
> {code}
> This fix alows to serialize and deserialize generic record with schema from the specified generated type.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)