[jira] [Commented] (FLINK-27255) Flink-avro does not support serialization and deserialization of avro schema longer than 65535 characters

["J Y (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-27255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522595#comment-17522595 ] 

J Y commented on FLINK-27255:
-----------------------------

has this issue existed for awhile now (1.11+)?  or, is it just constrained to 1.14.4?

> Flink-avro does not support serialization and deserialization of avro schema longer than 65535 characters
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-27255
>                 URL: https://issues.apache.org/jira/browse/FLINK-27255
>             Project: Flink
>          Issue Type: Bug
>          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>    Affects Versions: 1.14.4
>            Reporter: Haizhou Zhao
>            Assignee: Haizhou Zhao
>            Priority: Blocker
>
> The underlying serialization of avro schema uses string serialization method of ObjectOutputStream.class, however, the default string serialization by ObjectOutputStream.class does not support handling string of more than 66535 characters (64kb). As a result, constructing flink operators that input/output Avro Generic Record with huge schema is not possible.
>  
> The purposed fix is two change the serialization and deserialization method of these following classes so that huge string could also be handled.
>  
> [GenericRecordAvroTypeInfo|https://github.com/apache/flink/blob/master/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/typeutils/GenericRecordAvroTypeInfo.java#L107]
> [SerializableAvroSchema|https://github.com/apache/flink/blob/master/flink-formats/flink-avro/src/main/java/org/apache/flink/formats/avro/typeutils/SerializableAvroSchema.java#L55]
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)