You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Swinky Mann (Jira)" <ji...@apache.org> on 2021/09/21 20:05:00 UTC
[jira] [Commented] (SPARK-36819) DPP: Don't insert redundant
filters in case static partition pruning can be done
[ https://issues.apache.org/jira/browse/SPARK-36819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418300#comment-17418300 ]
Swinky Mann commented on SPARK-36819:
-------------------------------------
https://github.com/apache/spark/pull/34062
> DPP: Don't insert redundant filters in case static partition pruning can be done
> --------------------------------------------------------------------------------
>
> Key: SPARK-36819
> URL: https://issues.apache.org/jira/browse/SPARK-36819
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.2
> Reporter: Swinky Mann
> Priority: Minor
>
> Don't insert dynamic partition pruning filters in case the filters already referred statically. In case the filtering predicate on dimension table is in joinKey, no need to insert DPP filter in that case.
> DPP is not required in this Sample query:
> {{SELECT f.date_id, f.pid, f.sid FROM
> (select date_id, product_id as pid, store_id as sid from fact_stats) as f
> JOIN dim_stats s
> ON f.sid = s.store_id WHERE s.store_id = 3}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org