You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2016/03/13 03:07:33 UTC
[jira] [Assigned] (SPARK-13840) Disable Project Push Down Through
Filter
[ https://issues.apache.org/jira/browse/SPARK-13840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-13840:
------------------------------------
Assignee: (was: Apache Spark)
> Disable Project Push Down Through Filter
> ----------------------------------------
>
> Key: SPARK-13840
> URL: https://issues.apache.org/jira/browse/SPARK-13840
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Xiao Li
> Priority: Critical
>
> Now, ColumnPruning and PushPredicateThroughProject reverse each other's effect. Optimizer rules always reach the max iteration when optimizing some queries. Thus, we should disable columnPruning when hitting Filter for enabling predicate push down through project.
> The issue becomes worse when having another rule NullFiltering, which add extra Filters.
> In addition, columnPruning should not push Project through non-deterministic Filters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org