You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Timo Walther (Jira)" <ji...@apache.org> on 2020/10/23 14:51:00 UTC
[jira] [Updated] (FLINK-10211) Time indicators are not correctly
materialized for LogicalJoin
[ https://issues.apache.org/jira/browse/FLINK-10211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Timo Walther updated FLINK-10211:
---------------------------------
Description:
Currently {{org.apache.flink.table.calcite.RelTimeIndicatorConverter#visit(LogicalJoin)}} correctly handles only windowed joins. Output of non windowed joins shouldn't contain any time indicators.
A symptom of this is the exception:
{code}
org.apache.flink.table.api.TableException: Found more than one rowtime field: [orderTime, payTime] in the table that should be converted to a DataStream. Please select the rowtime field that should be used as event-time timestamp for the DataStream by casting all other fields to TIMESTAMP. at org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:906)
{code}
A long-term solution would be:
The root cause of this issue is the early phase in which {{RelTimeIndicatorConverter}} is called. Due to lack of information (since the join condition might not have been pushed into the join node), we can not differentiate between a window and non-window join. Thus, we cannot perform the time indicator materialization more fine grained. A solution would be to perform the materialization later after the logical optimization and before the physical translation, this would also make sense from a semantic perspective because time indicators are more a physical characteristic.
was:
Currently {{org.apache.flink.table.calcite.RelTimeIndicatorConverter#visit(LogicalJoin)}} correctly handles only windowed joins. Output of non windowed joins shouldn't contain any time indicators.
Update:
The root cause of this issue is the early phase in which {{RelTimeIndicatorConverter}} is called. Due to lack of information (since the join condition might not have been pushed into the join node), we can not differentiate between a window and non-window join. Thus, we cannot perform the time indicator materialization more fine grained. A solution would be to perform the materialization later after the logical optimization and before the physical translation, this would also make sense from a semantic perspective because time indicators are more a physical characteristic.
> Time indicators are not correctly materialized for LogicalJoin
> --------------------------------------------------------------
>
> Key: FLINK-10211
> URL: https://issues.apache.org/jira/browse/FLINK-10211
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / API, Table SQL / Planner
> Affects Versions: 1.6.0
> Reporter: Piotr Nowojski
> Priority: Major
>
> Currently {{org.apache.flink.table.calcite.RelTimeIndicatorConverter#visit(LogicalJoin)}} correctly handles only windowed joins. Output of non windowed joins shouldn't contain any time indicators.
> A symptom of this is the exception:
> {code}
> org.apache.flink.table.api.TableException: Found more than one rowtime field: [orderTime, payTime] in the table that should be converted to a DataStream. Please select the rowtime field that should be used as event-time timestamp for the DataStream by casting all other fields to TIMESTAMP. at org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:906)
> {code}
> A long-term solution would be:
> The root cause of this issue is the early phase in which {{RelTimeIndicatorConverter}} is called. Due to lack of information (since the join condition might not have been pushed into the join node), we can not differentiate between a window and non-window join. Thus, we cannot perform the time indicator materialization more fine grained. A solution would be to perform the materialization later after the logical optimization and before the physical translation, this would also make sense from a semantic perspective because time indicators are more a physical characteristic.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)