[jira] [Resolved] (SPARK-53074) Avoid partially clustered distribution when key-grouped partitioning used to satisfy non-JOIN required distribution

["Wenchen Fan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-53074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan resolved SPARK-53074.
---------------------------------
    Fix Version/s: 4.1.0
                   4.0.1
       Resolution: Fixed

Issue resolved by pull request 51818
[https://github.com/apache/spark/pull/51818]

> Avoid partially clustered distribution when key-grouped partitioning used to satisfy non-JOIN required distribution
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-53074
>                 URL: https://issues.apache.org/jira/browse/SPARK-53074
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.3.0, 3.4.0, 3.5.0, 4.1.0, 4.0.0
>            Reporter: Chirag Singh
>            Assignee: Chirag Singh
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.1.0, 4.0.1
>
>
> Currently, SPJ logic can apply partial clustering (when enabling) to either side of an inner JOIN as long as the nodes between the scan and JOIN preserve partitioning. This doesn't work if one of these nodes is using the scan's key-grouped partitioning to satisfy its required distribution (for example, a grouping agg or window function). 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org