You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by tejasapatil <gi...@git.apache.org> on 2017/03/14 23:47:16 UTC

[GitHub] spark pull request #16985: [SPARK-19122][SQL] Unnecessary shuffle+sort added...

Github user tejasapatil commented on a diff in the pull request:

    https://github.com/apache/spark/pull/16985#discussion_r106055916
  
    --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/Exchange.scala ---
    @@ -48,6 +49,10 @@ abstract class Exchange extends UnaryExecNode {
     case class ReusedExchangeExec(override val output: Seq[Attribute], child: Exchange)
       extends LeafExecNode {
     
    +  override def outputPartitioning: Partitioning = child.outputPartitioning
    --- End diff --
    
    @hvanhovell : continuing our discussion at https://github.com/apache/spark/pull/16985#discussion_r101905768 : 
    
    I found that `ReusedExchangeExec` does not use child's partitioning which was causing the tests to fail. This version of the PR is with your suggestion + works in all cases (all unit tests are passing).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org