[jira] [Created] (SPARK-9702) Repartition operator should use Exchange to perform its shuffle

["Josh Rosen (JIRA)" <ji...@apache.org>]

Josh Rosen created SPARK-9702:
---------------------------------

             Summary: Repartition operator should use Exchange to perform its shuffle
                 Key: SPARK-9702
                 URL: https://issues.apache.org/jira/browse/SPARK-9702
             Project: Spark
          Issue Type: Improvement
          Components: SQL
            Reporter: Josh Rosen


Spark SQL's {{Repartition}} operator is implemented in terms of Spark Core's repartition operator, which means that it has to perform lots of unnecessary row copying and inefficient row serialization. Instead, it would be better if this was implemented using some of Exchange's internals so that it can avoid row format conversions and generic getters / hashcodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org