You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Glauber M Dantas (Jira)" <ji...@apache.org> on 2024/05/17 13:50:00 UTC
[jira] [Updated] (FLINK-35392) SQL Lookup Join on external database failing when using 'FOR SYSTEM_TIME AS OF' statement
[ https://issues.apache.org/jira/browse/FLINK-35392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Glauber M Dantas updated FLINK-35392:
-------------------------------------
Description:
When transforming a DataStream to a Table and then using that table in a Join Lookup, an exception is raised: {{{}"Temporal table join currently only supports 'FOR SYSTEM_TIME AS OF' left table's time attribute field"{}}}.
From what I understood from the issue, it happens as the planner expects that the Tables part of the Join are registered previously on the Catalog.
The solution was to avoid using the DataStream to Table conversion:
```
Table kinesisStreamTbl = tblEnv.fromDataStream(
kinesisDataStream,
Schema.newBuilder()
// "proc_time" would be needed to make the lookup using 'FOR SYSTEM_TIME AS OF'; see: [https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/#processing-time-temporal-join]
.columnByExpression("proc_time", "PROCTIME()")
.build()
);
```
And instead create and use a Table registered on the Catalog:
```
tblEnv.createTable("KinesisInput", TableDescriptor.forConnector("kinesis")...);
```
If you need more details, you may access the discussion at: [https://stackoverflow.com/questions/78485099/sedona-flink-sql-lookup-on-external-database-failing-when-using-for-system-time]
was:
When transforming a DataStream to a Table and then using that table in a Join Lookup, an exception is raised: {{{}"Temporal table join currently only supports 'FOR SYSTEM_TIME AS OF' left table's time attribute field"{}}}.
From what I understood from the issue, it happens as the planner expects that the Tables part of the Join are registered previously on the Catalog.
The solution was to avoid using the DataStream to Table conversion:
```
Table kinesisStreamTbl = tblEnv.fromDataStream(
kinesisDataStream,
Schema.newBuilder()
// "proc_time" would be needed to make the lookup using 'FOR SYSTEM_TIME AS OF'; see: https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/#processing-time-temporal-join
.columnByExpression("proc_time", "PROCTIME()")
.build()
);
```
And instead create and use a Table registered on the Catalog:
```
tblEnv.createTable("KinesisInput", TableDescriptor.forConnector("kinesis")...);
```
If you need more details, you may access the discussion at: https://stackoverflow.com/questions/78485099/sedona-flink-sql-lookup-on-external-database-failing-when-using-for-system-time
> SQL Lookup Join on external database failing when using 'FOR SYSTEM_TIME AS OF' statement
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-35392
> URL: https://issues.apache.org/jira/browse/FLINK-35392
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 1.18.1
> Reporter: Glauber M Dantas
> Priority: Major
>
> When transforming a DataStream to a Table and then using that table in a Join Lookup, an exception is raised: {{{}"Temporal table join currently only supports 'FOR SYSTEM_TIME AS OF' left table's time attribute field"{}}}.
> From what I understood from the issue, it happens as the planner expects that the Tables part of the Join are registered previously on the Catalog.
> The solution was to avoid using the DataStream to Table conversion:
>
> ```
> Table kinesisStreamTbl = tblEnv.fromDataStream(
> kinesisDataStream,
> Schema.newBuilder()
> // "proc_time" would be needed to make the lookup using 'FOR SYSTEM_TIME AS OF'; see: [https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/queries/joins/#processing-time-temporal-join]
> .columnByExpression("proc_time", "PROCTIME()")
> .build()
> );
> ```
> And instead create and use a Table registered on the Catalog:
>
> ```
> tblEnv.createTable("KinesisInput", TableDescriptor.forConnector("kinesis")...);
> ```
>
> If you need more details, you may access the discussion at: [https://stackoverflow.com/questions/78485099/sedona-flink-sql-lookup-on-external-database-failing-when-using-for-system-time]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)