You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Yuanjian Li (Jira)" <ji...@apache.org> on 2019/08/22 03:00:01 UTC

[jira] [Created] (SPARK-28845) Enable spark.sql.execution.sortBeforeRepartition only for retried stages

Yuanjian Li created SPARK-28845:
-----------------------------------

             Summary: Enable spark.sql.execution.sortBeforeRepartition only for retried stages
                 Key: SPARK-28845
                 URL: https://issues.apache.org/jira/browse/SPARK-28845
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core, SQL
    Affects Versions: 3.0.0
            Reporter: Yuanjian Li


For fixing the correctness bug of SPARK-28699, we disable radix sort for the scenario of repartition in Spark SQL. This will cause a performance regression.

So for limiting the performance overhead, we'll do the optimizing work by only enable sort before repartition config for the retried stage. This work depends on SPARK-25341.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org