[jira] [Created] (FLINK-11184) Rework TableSource and TableSink interfaces

["Timo Walther (JIRA)" <ji...@apache.org>]

Timo Walther created FLINK-11184:
------------------------------------

             Summary: Rework TableSource and TableSink interfaces
                 Key: FLINK-11184
                 URL: https://issues.apache.org/jira/browse/FLINK-11184
             Project: Flink
          Issue Type: New Feature
          Components: Table API & SQL
            Reporter: Timo Walther
            Assignee: Timo Walther


There are a couple of shortcomings with the current {{TableSource}} and {{TableSink}} interface design. Some of the issues are covered in a [basic design document|https://docs.google.com/document/d/1Yaxp1UJUFW-peGLt8EIidwKIZEWrrA-pznWLuvaH39Y/edit#heading=h.41fd6rs7b3cf] that was published a while ago.

The design document has not been updated for some time and partially overlaps with the [current SQL DDL discussion|http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-SQL-DDL-Design-td25006.html] for the {{CREATE TABLE}} statement on the ML.

What needs to be solved:
- How to unify sources and sinks in regards of schema and time attributes?
- How to define watermarks, timestamp extractors or timestamp ingestion?
- How to define primary keys and partitioning keys?
- How to differentiate between update modes for tables (i.e. how to read from a append, retraction, or update table)?
- How to express all of the above without pulling in to many dependencies on other Flink modules if source and sink interfaces are located in {{flink-table-spi}} package?

As of the current state of the discussion, it seems that we might extend {{TableSchema}} to allow for returning the information above and remove current interfaces such as {{DefinedRowtimeAttribute}} or {{DefinedFieldMapping}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)