[jira] [Closed] (FLINK-28566) Adds materialization support to eliminate the non determinism generated by lookup join node

["lincoln lee (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-28566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

lincoln lee closed FLINK-28566.
-------------------------------
    Resolution: Abandoned

the new FLINK-28570 will cover this part, and no need to expose a new  'table.exec.lookup-join.upsert-materialize' option

> Adds materialization support to eliminate the non determinism generated by lookup join node
> -------------------------------------------------------------------------------------------
>
>                 Key: FLINK-28566
>                 URL: https://issues.apache.org/jira/browse/FLINK-28566
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / Planner
>            Reporter: lincoln lee
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.16.0
>
>
> In order to minimize the potential exceptions or data errors when many users use the update stream to lookup join an external 
> table (essentially due to the non-deterministic result based on processing-time to lookup external tables). 
> When update exists in the input stream and the lookup key does not contain the primary key of the external table,
> FLINK automatically adds materialization of the update by default, so that it will only lookup the external table 
> when the insert or update_after message arrives, and when the delete or update_before message arrives, it will 
> directly querying the latest version of the locally materialized data and sent it to downstream operator.
> To do so，we introduce a new option 'table.exec.lookup-join.upsert-materialize' and resue the `UpsertMaterialize`. By default, the materialize operator will be added when an update stream lookup an external table without containing its primary keys(includes no primary key defined). You can also choose no materialization(NONE) or force materialization(FORCE) which will always enable materialization except input is insert only.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)