You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Yin Huai (JIRA)" <ji...@apache.org> on 2016/01/20 21:50:39 UTC

[jira] [Commented] (SPARK-12850) Support bucket pruning (predicate pushdown for bucketed tables)

    [ https://issues.apache.org/jira/browse/SPARK-12850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109368#comment-15109368 ] 

Yin Huai commented on SPARK-12850:
----------------------------------

[~smilegator] Will you have time to work on this? We have supported writing bucketed tables, one optimization that we can make is to pruning those bucket files that we do not need based on the predicates of a query (like partition pruning). 

> Support bucket pruning (predicate pushdown for bucketed tables)
> ---------------------------------------------------------------
>
>                 Key: SPARK-12850
>                 URL: https://issues.apache.org/jira/browse/SPARK-12850
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Reynold Xin
>
> We now support bucketing. One optimization opportunity is to push some predicates into the scan to skip scanning files that definitely won't match the values.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org