[jira] [Commented] (FLINK-14716) Cooperate computed column with project push down

["Danny Chen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-14716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16972228#comment-16972228 ] 

Danny Chen commented on FLINK-14716:
------------------------------------

Thanks Jark for reporting this, i checked again the following rules:

* PushFilterIntoTableSourceScanRule // affected
* PushLimitIntoTableSourceScanRule // affected
* PushPartiitonIntoTableSourceScanRule
* PushProjectIntoTableSourceScanRule

The first 2 rules are affected by change FLINK-14665 if the source table is with computed column. The last 2 are still correct.

The DatabaseCalciteSchema returns a TableSourceTable when we use getTable from Calcite catalog reader, then this TableSourceTable was wrapped as a FlinkRelOptTable, this FlinkRelOptTable is the role to do the computed column conversion. This is the reason why i change the TableSourceTable row type to include the viarual columns.

In order to solve this, we may need to make sure what the TableSourceTable is(then we know that row type is should return), i figure out 2 ways:
# If we make sure that TableSourceTable return type should return virtual columns, then fixing the last 2 rules is okey.
# If we make sure that TableSourceTable return type should not include virtual columns, then we may need another "Table" abstraction to return a with virtual columns table when invoking DatabaseCalciteSchema#getTable

> Cooperate computed column with project push down
> ------------------------------------------------
>
>                 Key: FLINK-14716
>                 URL: https://issues.apache.org/jira/browse/FLINK-14716
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / Planner
>            Reporter: Jark Wu
>            Priority: Major
>
> In FLINK-14665, we supported computed column in planner. However, the implementation of {{TableSourceTable#getRowType}} will include computed columns which is possibly not right, because computed column is generated in the downstream Project, not in the source scan. 
> This is possibly lead to errors when push project into source scan which will call {{TableSourceTable#getRowType}} in the rule. Currently, all the tests are passed because we don't have a test to cover this, i.e. computed column on a projectable TableSource. We should at least add a test to cover this. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)