[jira] [Created] (FLINK-39258) FLIP-564: Support FROM_CHANGELOG and TO_CHANGELOG built-in PTFs

["Gustavo de Morais (Jira)" <ji...@apache.org>]

Gustavo de Morais created FLINK-39258:
-----------------------------------------

             Summary: FLIP-564: Support FROM_CHANGELOG and TO_CHANGELOG built-in PTFs
                 Key: FLINK-39258
                 URL: https://issues.apache.org/jira/browse/FLINK-39258
             Project: Flink
          Issue Type: New Feature
            Reporter: Gustavo de Morais
            Assignee: Gustavo de Morais


Flink's DataStream API offers flexible methods like toChangelogStream() and fromChangelogStream() to work with changelog streams, but SQL users currently lack this capability. 

This FLIP introduces two built-in Process Table Functions (PTFs) to bring _similar_ functionality with additional features to Flink SQL: 

- {*}FROM_CHANGELOG{*}: Converts an append-only stream of CDC records into a dynamic table, enabling, for example, custom CDC connector implementations directly in SQL. - {*}TO_CHANGELOG{*}: Converts a dynamic table back into an append-only changelog stream - the first operator that makes it possible to convert a retract/upsert stream back to append in SQL. 

Both PTFs will eventually support flexible operation mapping (e.g., Debezium-style 'c', 'u', 'd' codes), before/after image handling, configurable state TTL, and watermark-based ordering. They are designed to work symmetrically, so FROM_CHANGELOG(TO_CHANGELOG(table)) round-trips correctly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)