[jira] [Created] (SPARK-2255) create a self destructing iterator that release key value pairs from in-memory hash maps

["Reynold Xin (JIRA)" <ji...@apache.org>]

Reynold Xin created SPARK-2255:
----------------------------------

             Summary: create a self destructing iterator that release key value pairs from in-memory hash maps
                 Key: SPARK-2255
                 URL: https://issues.apache.org/jira/browse/SPARK-2255
             Project: Spark
          Issue Type: Improvement
          Components: SQL
            Reporter: Reynold Xin


This is a small thing to do that can help out with GC pressure. For aggregations (and potentially joins), we don't really need to hold onto the key value pairs as soon as we have iterate over them. We can create a self destructing iterator for AppendOnlyMap / ExternalAppendOnlyMap that removes references to the key value pair as the iterator goes through records so those memory can be freed quickly.





--
This message was sent by Atlassian JIRA
(v6.2#6252)