[jira] [Created] (FLINK-19694) Support [UPDATE_AFTER, DELETE] ChangelogMode for ScanTableSource

["Jark Wu (Jira)" <ji...@apache.org>]

Jark Wu created FLINK-19694:
-------------------------------

             Summary: Support [UPDATE_AFTER, DELETE] ChangelogMode for ScanTableSource
                 Key: FLINK-19694
                 URL: https://issues.apache.org/jira/browse/FLINK-19694
             Project: Flink
          Issue Type: Sub-task
          Components: Table SQL / API, Table SQL / Planner
            Reporter: Jark Wu
            Assignee: Jark Wu
             Fix For: 1.12.0


Currently, we only support insert-only or all-kinds (e.g. CDC format) ChangelogMode. In this issue, we want to support [UPDATE_AFTER, DELETE] ChangelogMode which indicates the source will emit only UPDATE_AFTER and DELETE messages during runtime (e.g. an upsert source). The planner will add the a materialization operator when the ChangelogMode of the source is [UPDATE_AFTER, DELETE].  

The materialization operator will materialize the upsert stream and generate changelog stream with full change messages. In the physical operator, we will use state to know whether the key is the first time to be seen. The operator will produce INSERT rows, or additionally generate UPDATE_BEFORE rows for the previous image, or produce DELETE rows with all columns filled with values.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)