[jira] [Closed] (FLINK-7696) Add projection push-down support for TableSources with time attributes

["Fabian Hueske (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fabian Hueske closed FLINK-7696.
--------------------------------
       Resolution: Implemented
    Fix Version/s: 1.4.0

Implemented for 1.4.0 with 9a2ba6e058e907aef65e0af8731ca5ec8733712e

> Add projection push-down support for TableSources with time attributes
> ----------------------------------------------------------------------
>
>                 Key: FLINK-7696
>                 URL: https://issues.apache.org/jira/browse/FLINK-7696
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table API & SQL
>    Affects Versions: 1.4.0
>            Reporter: Fabian Hueske
>             Fix For: 1.4.0
>
>
> Table sources that implement the {{DefinedProctimeAttribute}} or {{DefinedRowtimeAttribute}} do not support projection push-down even if they also implement {{ProjectableTableSource}}. 
> There are several problems:
> - the schema of a {{TableSource}} that implements {{DefinedRowtimeAttribute}} or {{DefinedProctimeAttribute}} is constructed in the catalog not in the {{TableSource}} (proctime fields are always appended at the end).
> - the {{ProjectableTableSource.projectFields()}} method returns the projected fields as int indicies. In order to handle the indicies correctly, the TableSource would need to know about the internals of the Table API.
> - {{ProjectableTableSource.projectFields()}} might reorder fields and move a proctime field into the middle of the schema. However, the TableSource has no control over that.
> - A {{TableSource}} that implements {{DefinedRowtimeAttribute}} or {{DefinedProctimeAttribute}} would need to change the return values of {{getRowtimeAttribute()}} or {{getProctimeAttribute()}} depending on whether the attribute is kept or not.
> Adjusting the schema of table sources inside of the Table API makes all of this quite messy. Maybe we need to refine the interfaces. For instance, we could ask users to explicitly add time indicator fields in the {{TypeInformation}} returned by {{TableSource.getReturnType()}}. However, that might collide with plans to add computable time attributes as proposed in FLINK-7548.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)