[jira] [Commented] (FLINK-20175) Avro Confluent Registry SQL format does not support adding nullable columns

["Dawid Wysakowicz (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-20175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233627#comment-17233627 ] 

Dawid Wysakowicz commented on FLINK-20175:
------------------------------------------

I included a fix as part of this PR: https://github.com/apache/flink/pull/14085

> Avro Confluent Registry SQL format does not support adding nullable columns
> ---------------------------------------------------------------------------
>
>                 Key: FLINK-20175
>                 URL: https://issues.apache.org/jira/browse/FLINK-20175
>             Project: Flink
>          Issue Type: Bug
>          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / Ecosystem
>            Reporter: Dawid Wysakowicz
>            Assignee: Dawid Wysakowicz
>            Priority: Critical
>             Fix For: 1.12.0
>
>
> The {{AvroSchemaConverter#convertToSchema}} generates a union with "null" for nullable logical types, but it does not set the default value to null. In turn it makes it impossible to generate a backwards compatible schema from a DDL statement.
> Example:
> 1. Create a table: {{CREATE TABLE t (id INT NOT NULL) WITH (/* avro confluent format*/)}}
> 2. Create a new table over the same topic or alter the old table with {{CREATE TABLE newT(id INT NOT NULL, optionalDescription STRING) WITH (/* avro confluent format */)}}
> 3. When reading from {{newT}} records inserted into {{t}} it will fail, because the {{optionalDescription}} has no default value.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)