You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "XiDuo You (Jira)" <ji...@apache.org> on 2021/07/02 09:59:00 UTC
[jira] [Updated] (SPARK-35989) Do not remove REPARTITION_BY_NUM
shuffle if AQE is enabled
[ https://issues.apache.org/jira/browse/SPARK-35989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
XiDuo You updated SPARK-35989:
------------------------------
Description: The shuffle origin is `REPARTITION_BY_NUM` if user specify an exact partition number with repartition, then we should not do any change of the number. That said, the shuffle output partitioning number should be always same with user expected.
> Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled
> ----------------------------------------------------------
>
> Key: SPARK-35989
> URL: https://issues.apache.org/jira/browse/SPARK-35989
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: XiDuo You
> Priority: Major
>
> The shuffle origin is `REPARTITION_BY_NUM` if user specify an exact partition number with repartition, then we should not do any change of the number. That said, the shuffle output partitioning number should be always same with user expected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org