You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xiao Li (JIRA)" <ji...@apache.org> on 2016/03/13 02:58:33 UTC

[jira] [Created] (SPARK-13840) Disable Project Push Down Through Filter

Xiao Li created SPARK-13840:
-------------------------------

             Summary: Disable Project Push Down Through Filter
                 Key: SPARK-13840
                 URL: https://issues.apache.org/jira/browse/SPARK-13840
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.0.0
            Reporter: Xiao Li
            Priority: Critical


Now, ColumnPruning and PushPredicateThroughProject reverse each other's effect. Optimizer rules always reach the max iteration when optimizing some queries. Thus, we should disable columnPruning when hitting Filter for enabling predicate push down through project. 

The issue becomes worse when having another rule NullFiltering, which add extra Filters. 

In addition, columnPruning should not push Project through non-deterministic Filters. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org