[jira] [Closed] (FLINK-40264) nested legacyTimestampMapping

["Gyula Fora (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-40264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gyula Fora closed FLINK-40264.
------------------------------
    Fix Version/s: 2.4.0
       Resolution: Fixed

merged to master 40d486e2b6465680ed93305ebc51bcc87b26b995

> nested legacyTimestampMapping
> -----------------------------
>
>                 Key: FLINK-40264
>                 URL: https://issues.apache.org/jira/browse/FLINK-40264
>             Project: Flink
>          Issue Type: Bug
>          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
>    Affects Versions: 1.20.0, 2.2.0, 2.3.0
>            Reporter: Pritam Kumar
>            Assignee: Pritam Kumar
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.4.0
>
>
> h2. Problem
>   {{AvroToRowDataConverters#createConverter}} dispatches a nested {{ROW}} to the single argument overload:
>   {code:java}
>   case ROW:
>       return createRowConverter((RowType) type);
>   {code}
>   {{createRowConverter(RowType)}} defaults {{legacyTimestampMapping}} to {{true}}, so the flag is dropped for every nesting level below the top one. Since {{TIMESTAMP_LTZ}} is only supported by the non legacy mapping, a nested {{TIMESTAMP_LTZ}}
>   column fails even when the non legacy mapping was explicitly requested:
>   {noformat}
>   java.lang.UnsupportedOperationException: Unsupported type: TIMESTAMP_LTZ(3) NOT NULL
>   {noformat}
>   The failure happens while the deserialization schema is being constructed, so the job never starts.
>   h2. Reproduction
>   {code:java}
>   DataType dataType =
>           ROW(
>                   FIELD("id", INT().notNull()),
>                   FIELD("nested", ROW(FIELD("ltz3", TIMESTAMP_LTZ(3).notNull())).notNull()))
>           .notNull();
>   RowType rowType = (RowType) dataType.getLogicalType();
>   new AvroRowDataDeserializationSchema(
>           rowType,
>           InternalTypeInfo.of(rowType),
>           AvroEncoding.BINARY,
>           false /* legacyTimestampMapping */);   // throws
>   {code}
>   {{AvroSchemaConverter#convertToSchema}} handles the same type correctly with {{legacyTimestampMapping = false}}, and {{RowDataToAvroConverters}} propagates the flag properly, so only the deserialization side is affected.
>   h2. Fix
>   Propagate the flag:
>   {code:java}
>   case ROW:
>       return createRowConverter((RowType) type, legacyTimestampMapping);
>   {code}
>   Strictly a widening: the only code paths whose behaviour changes are ones that throw today.
>   h2. Introduced by
>   FLINK-33198, first released in 1.19.0.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)