You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Grzegorz Kołakowski (Jira)" <ji...@apache.org> on 2025/03/04 15:20:00 UTC
[jira] [Commented] (FLINK-27641) Create view lost Time attribute in Hive Catalog
[ https://issues.apache.org/jira/browse/FLINK-27641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932349#comment-17932349 ]
Grzegorz Kołakowski commented on FLINK-27641:
---------------------------------------------
For missing *PROCTIME* attribute we can easily do a workaround ("from (select *, proctime() as `proctime` ...)" subquery). The real problem is missing *ROWTIME* attribute which prevents from running all SQL constructs that rely on watermarking.
> Create view lost Time attribute in Hive Catalog
> -----------------------------------------------
>
> Key: FLINK-27641
> URL: https://issues.apache.org/jira/browse/FLINK-27641
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Affects Versions: 1.12.3, 1.14.4
> Reporter: Robert Wu
> Priority: Minor
>
> Create table in hive catalog with the following sql state. :
>
> {code:java}
> CREATE TABLE user_score(
> username varchar,
> score varchar,
> proctime AS PROCTIME()
> ) with (
> 'connector'='datagen',
> 'rows-per-second'='2',
> 'fields.score.length'='2',
> 'fields.username.length'='2'
> );{code}
>
> We can get the description:
>
> {code:java}
> DESCRIBE user_score;
> +----------+-----------------------------+-------+-----+---------------+-----------+
> | name | type | null | key | extras | watermark |
> +----------+-----------------------------+-------+-----+---------------+-----------+
> | username | STRING | true | | | |
> | score | STRING | true | | | |
> | proctime | TIMESTAMP_LTZ(3) *PROCTIME* | false | | AS PROCTIME() | |
> +----------+-----------------------------+-------+-----+---------------+-----------+
> {code}
>
>
> However,view create in hive catalog will lost Time attribute in the proctime field:
>
> {code:java}
> create view view_score_hive_catalog as select * from user_score;{code}
>
> {code:java}
> DESCRIBE view_score_hive_catalog;
>
> +----------+------------------+-------+-----+--------+-----------+
> | name | type | null | key | extras | watermark |
> +----------+------------------+-------+-----+--------+-----------+
> | username | STRING | true | | | |
> | score | STRING | true | | | |
> | proctime | TIMESTAMP_LTZ(3) | false | | | |
> +----------+------------------+-------+-----+--------+-----------+
> {code}
>
>
> Otherwise,when we excute the same state. in default catalog, things are going to change:
> {code:java}
> Create view view_score_mem_catalog as select * from myhive.[hive_database].user_score;{code}
>
> {code:java}
> DESCRIBE view_score_mem_catalog;
> +----------+-----------------------------+-------+-----+--------+-----------+
> | name | type | null | key | extras | watermark |
> +----------+-----------------------------+-------+-----+--------+-----------+
> | username | STRING | true | | | |
> | score | STRING | true | | | |
> | proctime | TIMESTAMP_LTZ(3) *PROCTIME* | false | | | |
> +----------+-----------------------------+-------+-----+--------+-----------+
> {code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)