You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wenchen Fan (Jira)" <ji...@apache.org> on 2022/03/10 05:08:00 UTC
[jira] [Assigned] (SPARK-38148) Do not add dynamic partition pruning if there exists static partition pruning
[ https://issues.apache.org/jira/browse/SPARK-38148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan reassigned SPARK-38148:
-----------------------------------
Assignee: XiDuo You
> Do not add dynamic partition pruning if there exists static partition pruning
> -----------------------------------------------------------------------------
>
> Key: SPARK-38148
> URL: https://issues.apache.org/jira/browse/SPARK-38148
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: XiDuo You
> Assignee: XiDuo You
> Priority: Major
>
> Dynamic partition pruning add a filter as long as the join condition contains partition columns. But if there exists other condition which contains the static partition pruning, it's unnecessary to add an extra dynamic partition pruning.
> For example:
> {code:java}
> CREATE TABLE t1 (c1 int) USING PARQUET PARTITIONED BY (p1 string);
> CREATE TABLE t2 (c2 int) USING PARQUET PARTITIONED BY (p2 string);
> SELECT * FROM t1 JOIN t2 ON t1.p1 = t2.p2 and t1.p1 = 'a' AND t2.c2 > 0;
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org