You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Josh Rosen (JIRA)" <ji...@apache.org> on 2015/06/12 02:44:00 UTC

[jira] [Created] (SPARK-8319) Enable Tungsten shuffle manager for some shuffles that specify key orderings

Josh Rosen created SPARK-8319:
---------------------------------

             Summary: Enable Tungsten shuffle manager for some shuffles that specify key orderings
                 Key: SPARK-8319
                 URL: https://issues.apache.org/jira/browse/SPARK-8319
             Project: Spark
          Issue Type: Improvement
          Components: Shuffle
            Reporter: Josh Rosen
            Assignee: Josh Rosen


The Tungsten ShuffleManager falls back to regular SortShuffleManager whenever the shuffle dependency specifies a key ordering, but technically we only need to fall back when an aggregator is also specified.  We should update the fallback logic to handle this case so that the Tungsten optimizations can apply to more workloads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org