You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2020/11/16 20:24:00 UTC
[jira] [Comment Edited] (SPARK-33183) Bug in optimizer rule
EliminateSorts
[ https://issues.apache.org/jira/browse/SPARK-33183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233058#comment-17233058 ]
Dongjoon Hyun edited comment on SPARK-33183 at 11/16/20, 8:23 PM:
------------------------------------------------------------------
I added SPARK-23973 as "is caused by". If then, Apache Spark 2.3 seems to be okay. Please let me know if this affects older Sparks.
was (Author: dongjoon):
I added SPARK-23973 as "is caused by". If then, Apache Spark 2.3 seems to be okay.
> Bug in optimizer rule EliminateSorts
> ------------------------------------
>
> Key: SPARK-33183
> URL: https://issues.apache.org/jira/browse/SPARK-33183
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.4.5, 2.4.6, 2.4.7, 2.4.8, 3.0.2, 3.1.0
> Reporter: Allison Wang
> Assignee: Allison Wang
> Priority: Major
> Labels: correctness
> Fix For: 2.4.8, 3.0.2, 3.1.0
>
>
> Currently, the rule {{EliminateSorts}} removes a global sort node if its child plan already satisfies the required sort order without checking if the child plan's ordering is local or global. For example, in the following scenario, the first sort shouldn't be removed because it has a stronger guarantee than the second sort even if the sort orders are the same for both sorts.
> {code:java}
> Sort(orders, global = True, ...)
> Sort(orders, global = False, ...){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org