[jira] [Commented] (SPARK-9489) Remove compatibleWith, meetsRequirements, and needsAnySort checks from Exchange

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-9489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648405#comment-14648405 ] 

Apache Spark commented on SPARK-9489:
-------------------------------------

User 'JoshRosen' has created a pull request for this issue:
https://github.com/apache/spark/pull/7807

> Remove compatibleWith, meetsRequirements, and needsAnySort checks from Exchange
> -------------------------------------------------------------------------------
>
>                 Key: SPARK-9489
>                 URL: https://issues.apache.org/jira/browse/SPARK-9489
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Josh Rosen
>            Assignee: Josh Rosen
>
> While reviewing [~yhuai]'s patch for SPARK-2205, I noticed that Exchange's {{compatible}} check may be incorrectly returning {{false}} in many cases.  As far as I know, this is not actually a problem because the {{compatible}}, {{meetsRequirements}}, and {{needsAnySort}} checks are serving only as short-circuit performance optimizations that are not necessary for correctness.
> In order to reduce code complexity, I think that we should remove these checks and unconditionally rewrite the operator's children.  This should be safe because we rewrite the tree in a single bottom-up pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org