You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Christiaan Biesterbosch (Jira)" <ji...@apache.org> on 2025/08/22 13:07:00 UTC
[jira] [Updated] (NIFI-14897) org.apache.nifi.serialization.RecordSetWriter does not support the Avro Fixed type
[ https://issues.apache.org/jira/browse/NIFI-14897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christiaan Biesterbosch updated NIFI-14897:
-------------------------------------------
Description:
See [this commit for a failing test|[https://github.com/apache/nifi/commit/b11eb5912e651e0cbe5bc32c9b228c58071b477a|https://github.com/apache/nifi/commit/b11eb5912e651e0cbe5bc32c9b228c58071b477a).]].
The problem seems to be that the Avro library expects something implementing `GenericFixed` for the Fixed type, but `RecordSetWriter` creates a `java.nio.HeapByteBuffer`.
{{A workaround is using the Avro Bytes type, but that doesn't work with `"logicalType": "uuid"`.}}
{{-------------------------------------------------------------------------------}}
{{Test set: org.apache.nifi.avro.TestWriteAvroResultWithSchema}}
{{-------------------------------------------------------------------------------}}
{{Tests run: 7, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.096 s <<< FAILURE! – in org.apache.nifi.avro.TestWriteAvroResultWithSchema}}
{{org.apache.nifi.avro.TestWriteAvroResultWithSchema.testDataTypes – Time elapsed: 0.003 s <<< ERROR!}}
{{org.apache.avro.file.DataFileWriter$AppendWriteException: java.lang.ClassCastException: value java.nio.HeapByteBuffer[pos=0 lim=16 cap=16] (a java.nio.HeapByteBuffer) cannot be cast to expected type fixed16 at data_types.fixed}}
{{at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:326)}}
{{at org.apache.nifi.avro.WriteAvroResultWithSchema.writeRecord(WriteAvroResultWithSchema.java:61)}}
{{at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)}}
{{at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:52)}}
{{at org.apache.nifi.avro.TestWriteAvroResult.testDataTypes(TestWriteAvroResult.java:316)}}
{{at java.base/java.lang.reflect.Method.invoke(Method.java:565)}}
{{at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)}}
{{at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)}}
{{Caused by: java.lang.ClassCastException: value java.nio.HeapByteBuffer[pos=0 lim=16 cap=16] (a java.nio.HeapByteBuffer) cannot be cast to expected type fixed16 at data_types.fixed}}
{{at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:79)}}
{{at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:30)}}
{{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:84)}}
{{at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:323)}}
{{... 7 more}}
{{Caused by: java.lang.ClassCastException: class java.nio.HeapByteBuffer cannot be cast to class org.apache.avro.generic.GenericFixed (java.nio.HeapByteBuffer is in module java.base of loader 'bootstrap'; org.apache.avro.generic.GenericFixed is in unnamed module of loader 'app')}}
{{at org.apache.avro.generic.GenericDatumWriter.writeFixed(GenericDatumWriter.java:408)}}
{{at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:162)}}
{{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:95)}}
{{at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:245)}}
{{at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:234)}}
{{at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:145)}}
{{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:95)}}
{{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:82)}}
{{... 8 more}}
was:
See [this commit for a failing test]([https://github.com/apache/nifi/commit/b11eb5912e651e0cbe5bc32c9b228c58071b477a).]
The problem seems to be that the Avro library expects something implementing `GenericFixed` for the Fixed type, but `RecordSetWriter` creates a `java.nio.HeapByteBuffer`.
A workaround is using the Avro Bytes type, but that doesn't work with `"logicalType": "uuid"`.
```
-------------------------------------------------------------------------------
Test set: org.apache.nifi.avro.TestWriteAvroResultWithSchema
-------------------------------------------------------------------------------
Tests run: 7, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.096 s <<< FAILURE! -- in org.apache.nifi.avro.TestWriteAvroResultWithSchema
org.apache.nifi.avro.TestWriteAvroResultWithSchema.testDataTypes -- Time elapsed: 0.003 s <<< ERROR!
org.apache.avro.file.DataFileWriter$AppendWriteException: java.lang.ClassCastException: value java.nio.HeapByteBuffer[pos=0 lim=16 cap=16] (a java.nio.HeapByteBuffer) cannot be cast to expected type fixed16 at data_types.fixed
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:326)
at org.apache.nifi.avro.WriteAvroResultWithSchema.writeRecord(WriteAvroResultWithSchema.java:61)
at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:52)
at org.apache.nifi.avro.TestWriteAvroResult.testDataTypes(TestWriteAvroResult.java:316)
at java.base/java.lang.reflect.Method.invoke(Method.java:565)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
Caused by: java.lang.ClassCastException: value java.nio.HeapByteBuffer[pos=0 lim=16 cap=16] (a java.nio.HeapByteBuffer) cannot be cast to expected type fixed16 at data_types.fixed
at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:79)
at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:30)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:84)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:323)
... 7 more
Caused by: java.lang.ClassCastException: class java.nio.HeapByteBuffer cannot be cast to class org.apache.avro.generic.GenericFixed (java.nio.HeapByteBuffer is in module java.base of loader 'bootstrap'; org.apache.avro.generic.GenericFixed is in unnamed module of loader 'app')
at org.apache.avro.generic.GenericDatumWriter.writeFixed(GenericDatumWriter.java:408)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:162)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:95)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:245)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:234)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:145)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:95)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:82)
... 8 more
```
> org.apache.nifi.serialization.RecordSetWriter does not support the Avro Fixed type
> ----------------------------------------------------------------------------------
>
> Key: NIFI-14897
> URL: https://issues.apache.org/jira/browse/NIFI-14897
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.5.0
> Reporter: Christiaan Biesterbosch
> Priority: Major
>
> See [this commit for a failing test|[https://github.com/apache/nifi/commit/b11eb5912e651e0cbe5bc32c9b228c58071b477a|https://github.com/apache/nifi/commit/b11eb5912e651e0cbe5bc32c9b228c58071b477a).]].
> The problem seems to be that the Avro library expects something implementing `GenericFixed` for the Fixed type, but `RecordSetWriter` creates a `java.nio.HeapByteBuffer`.
> {{A workaround is using the Avro Bytes type, but that doesn't work with `"logicalType": "uuid"`.}}
> {{-------------------------------------------------------------------------------}}
> {{Test set: org.apache.nifi.avro.TestWriteAvroResultWithSchema}}
> {{-------------------------------------------------------------------------------}}
> {{Tests run: 7, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.096 s <<< FAILURE! – in org.apache.nifi.avro.TestWriteAvroResultWithSchema}}
> {{org.apache.nifi.avro.TestWriteAvroResultWithSchema.testDataTypes – Time elapsed: 0.003 s <<< ERROR!}}
> {{org.apache.avro.file.DataFileWriter$AppendWriteException: java.lang.ClassCastException: value java.nio.HeapByteBuffer[pos=0 lim=16 cap=16] (a java.nio.HeapByteBuffer) cannot be cast to expected type fixed16 at data_types.fixed}}
> {{at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:326)}}
> {{at org.apache.nifi.avro.WriteAvroResultWithSchema.writeRecord(WriteAvroResultWithSchema.java:61)}}
> {{at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)}}
> {{at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:52)}}
> {{at org.apache.nifi.avro.TestWriteAvroResult.testDataTypes(TestWriteAvroResult.java:316)}}
> {{at java.base/java.lang.reflect.Method.invoke(Method.java:565)}}
> {{at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)}}
> {{at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)}}
> {{Caused by: java.lang.ClassCastException: value java.nio.HeapByteBuffer[pos=0 lim=16 cap=16] (a java.nio.HeapByteBuffer) cannot be cast to expected type fixed16 at data_types.fixed}}
> {{at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:79)}}
> {{at org.apache.avro.path.TracingClassCastException.summarize(TracingClassCastException.java:30)}}
> {{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:84)}}
> {{at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:323)}}
> {{... 7 more}}
> {{Caused by: java.lang.ClassCastException: class java.nio.HeapByteBuffer cannot be cast to class org.apache.avro.generic.GenericFixed (java.nio.HeapByteBuffer is in module java.base of loader 'bootstrap'; org.apache.avro.generic.GenericFixed is in unnamed module of loader 'app')}}
> {{at org.apache.avro.generic.GenericDatumWriter.writeFixed(GenericDatumWriter.java:408)}}
> {{at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:162)}}
> {{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:95)}}
> {{at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:245)}}
> {{at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:234)}}
> {{at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:145)}}
> {{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:95)}}
> {{at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:82)}}
> {{... 8 more}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)