[jira] [Commented] (FLINK-30292) Better support for conversion between DataType and TypeInformation

["Timo Walther (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-30292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17648563#comment-17648563 ] 

Timo Walther commented on FLINK-30292:
--------------------------------------

Thanks for starting this discussion [~yunfengzhou]. I agree that the current approach using `ExternalTypeInfo` adds some serialization overhead. It was introduced to eliminate any friction during conversion of both type systems during conversion Table API->DataStream API. This purpose was successful. However, we can further evolve `ExternalTypeInfo` and use more specialized type info/serializers. In theory, if the `DataType` is of class `Row` with default conversions in every subfield, `ExternalTypeInfo.of` can also return `RowTypeInfo` with `BasicTypeInfo.*` etc. This specialized logic has just not been implemented yet.

In any case, there were already discussion about introducing a dedicated `table-types` module to ease the development and expose common utilities including the data structure converters.

> Better support for conversion between DataType and TypeInformation
> ------------------------------------------------------------------
>
>                 Key: FLINK-30292
>                 URL: https://issues.apache.org/jira/browse/FLINK-30292
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / API
>    Affects Versions: 1.15.3
>            Reporter: Yunfeng Zhou
>            Priority: Major
>
> In Flink 1.15, we have the following ways to convert a DataType to a TypeInformation. Each of them has some disadvantages.
> * `TypeConversions.fromDataTypeToLegacyInfo`
> It might lead to precision losses in face of some data types like timestamp.
> It has been deprecated.
> * `ExternalTypeInfo.of`
> It cannot be used to get detailed type information like `RowTypeInfo`
> It might bring some serialization overhead.
> Given that the ways mentioned above are both not perfect,  Flink SQL should provide a better API to support DataType-TypeInformation conversions, and thus better support Table-DataStream conversions.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)