[jira] [Commented] (FLINK-33353) SQL fails because "TimestampType.kind" is not serialized

["Ferenc Csaky (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-33353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17782089#comment-17782089 ] 

Ferenc Csaky commented on FLINK-33353:
--------------------------------------

[~twalthr] any thoughts?

> SQL fails because "TimestampType.kind" is not serialized 
> ---------------------------------------------------------
>
>                 Key: FLINK-33353
>                 URL: https://issues.apache.org/jira/browse/FLINK-33353
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / API
>    Affects Versions: 1.18.0
>            Reporter: Ferenc Csaky
>            Priority: Major
>
> We have a custom persistent catalog store, which stores tables, views etc. in a DB. In our application, it is required to utilize the serialized formats of entities, but the same applies to the Hive, as it functions as a persistent catalog.
> Take the following example SQL:
> {code:sql}
> CREATE TABLE IF NOT EXISTS `txn_gen` (
>   `txn_id` INT,
>   `amount` INT,
>   `ts` TIMESTAMP(3),
>    WATERMARK FOR `ts` AS `ts` - INTERVAL '1' SECOND
> ) WITH (
>   'connector' = 'datagen',
>   'fields.txn_id.min' = '1',
>   'fields.txn_id.max' = '5',
>   'rows-per-second' = '1'
> );
> CREATE VIEW IF NOT EXISTS aggr_ten_sec AS
>   SELECT txn_id,
>          TUMBLE_ROWTIME(`ts`, INTERVAL '10' SECOND) AS w_row_time,
>          COUNT(txn_id) AS txn_count
>     FROM txn_gen
>     GROUP BY txn_id, TUMBLE(`ts`, INTERVAL '10' SECOND);
> SELECT txn_id,
>        SUM(txn_count),
>        TUMBLE_START(w_row_time, INTERVAL '20' SECOND) AS total_txn_count
>   FROM aggr_ten_sec
>   GROUP BY txn_id, TUMBLE(w_row_time, INTERVAL '20' SECOND);
> {code}
> This will work without any problems when we simply execute it in a {{TableEnvironment}}, but it fails with the below error when we try to execute the query based on the serialized table metadata.
> {code}
> org.apache.flink.table.api.TableException: Window aggregate can only be defined over a time attribute column, but TIMESTAMP(3) encountered.
> {code}
> If there is a view which would require to use ROWTIME, it will be lost and we cannot recreate the same query from the serialized entites.
> Currently in [TimestampType|https://github.com/apache/flink/blob/master/flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/TimestampType.java] the "kind" field is deliberatly annotated as {{@Internal}} and is not serialized, although it breaks this functionality.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)