You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xiao Li (JIRA)" <ji...@apache.org> on 2016/03/13 02:58:33 UTC
[jira] [Created] (SPARK-13840) Disable Project Push Down Through
Filter
Xiao Li created SPARK-13840:
-------------------------------
Summary: Disable Project Push Down Through Filter
Key: SPARK-13840
URL: https://issues.apache.org/jira/browse/SPARK-13840
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.0.0
Reporter: Xiao Li
Priority: Critical
Now, ColumnPruning and PushPredicateThroughProject reverse each other's effect. Optimizer rules always reach the max iteration when optimizing some queries. Thus, we should disable columnPruning when hitting Filter for enabling predicate push down through project.
The issue becomes worse when having another rule NullFiltering, which add extra Filters.
In addition, columnPruning should not push Project through non-deterministic Filters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org