You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Xiangrui Meng (JIRA)" <ji...@apache.org> on 2014/10/24 23:21:35 UTC

[jira] [Created] (SPARK-4084) Reuse sort key in Sorter

Xiangrui Meng created SPARK-4084:
------------------------------------

             Summary: Reuse sort key in Sorter
                 Key: SPARK-4084
                 URL: https://issues.apache.org/jira/browse/SPARK-4084
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
            Reporter: Xiangrui Meng


Sorter uses generic-typed key for sorting. When data is large, it creates lots of key objects, which is not efficient. We should reuse the key in Sorter for memory efficiency. This change is part of the petabyte sort implementation from [~rxin].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org