[jira] [Created] (SPARK-21941) Stop storing unused attemptId in SQLTaskMetrics

["Andrew Ash (JIRA)" <ji...@apache.org>]

Andrew Ash created SPARK-21941:
----------------------------------

             Summary: Stop storing unused attemptId in SQLTaskMetrics
                 Key: SPARK-21941
                 URL: https://issues.apache.org/jira/browse/SPARK-21941
             Project: Spark
          Issue Type: Bug
          Components: Web UI
    Affects Versions: 2.2.0
            Reporter: Andrew Ash


Currently SQLTaskMetrics has a long attemptId field on it that is unused, with a TODO saying to populate the value in the future.  We should save this memory by leaving the TODO but taking the unused field out of the class.

I have a driver that heap dumped on OOM and has 390,105 instances of SQLTaskMetric -- removing this 8 bytes field will save roughly 390k*8 = 3.1MB of heap space.  It's not going to fix my OOM, but there's no reason to put this pressure on the GC if we don't get anything by storing it.

https://github.com/apache/spark/blob/v2.2.0/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala#L485



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org