[jira] [Created] (FLINK-16987) Add new table source and sink interfaces

["Timo Walther (Jira)" <ji...@apache.org>]

Timo Walther created FLINK-16987:
------------------------------------

             Summary: Add new table source and sink interfaces
                 Key: FLINK-16987
                 URL: https://issues.apache.org/jira/browse/FLINK-16987
             Project: Flink
          Issue Type: New Feature
          Components: Table SQL / API, Table SQL / Planner
            Reporter: Timo Walther


Proper support for handling changelogs, more efficient processing of data through the new Blink planner, and unified interfaces that are DataStream API agnostic make it necessary to rework the table source and sink interfaces.

The goals of this FLIP are:
 * *Simplify the current interface architecture*:
 * Merge upsert, retract, and append sinks.
 * Unify batch and streaming sources.
 * Unify batch and streaming sinks.


 * *Allow sources to produce a changelog*:
 * UpsertTableSources have been requested a lot by users. Now is the time to open the internal planner capabilities via the new interfaces.
 * According to FLIP-105, we would like to support changelogs for processing formats such as [Debezium|https://debezium.io/].


 * *Don't rely on DataStream API for source and sinks*:
 * According to FLIP-32, the Table API and SQL should be independent of the DataStream API which is why the `table-common` module has no dependencies on `flink-streaming-java`.
 * Source and sink implementations should only depend on the `table-common` module after FLIP-27.
 * Until FLIP-27 is ready, we still put most of the interfaces in `table-common` and strictly separate interfaces that communicate with a planner and actual runtime reader/writers.


 * *Implement efficient sources and sinks without planner dependencies*:
 * Make Blink's internal data structures available to connectors.
 * Introduce stable interfaces for data structures that can be marked as `@PublicEvolving`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)