[jira] [Commented] (FLINK-5859) support partition pruning on Table API & SQL

["Fabian Hueske (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875689#comment-15875689 ] 

Fabian Hueske commented on FLINK-5859:
--------------------------------------

Partition pruning is a kind of coarse-grained filter push-down.

I think this issue should be solved by FLINK-3849 which tries to push filters into {{TableSources}}. 
It the responsibility of {{TableSources}} that implement {{FilterableTableSource}} to skip irrelevant input.

What do you think [~godfreyhe]?

> support partition pruning on Table API & SQL
> --------------------------------------------
>
>                 Key: FLINK-5859
>                 URL: https://issues.apache.org/jira/browse/FLINK-5859
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>            Reporter: godfrey he
>            Assignee: godfrey he
>
> Many data sources are partitionable storage, e.g. HDFS, Druid. And many queries just need to read a small subset of the total data. We can use partition information to prune or skip over files irrelevant to the user’s queries. Both query optimization time and execution time can be reduced obviously, especially for a large partitioned table.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)