[jira] [Commented] (SPARK-11192) Spark sql seems to leak org.apache.spark.sql.execution.ui.SQLTaskMetrics objects over time

["Josh Rosen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965371#comment-14965371 ] 

Josh Rosen commented on SPARK-11192:
------------------------------------

Hi [~blivingston], this sounds like it might be a duplicate of SPARK-11126, which will be fixed in 1.5.2. Does the description of that issue match up with the behavior that you observed?

> Spark sql seems to leak org.apache.spark.sql.execution.ui.SQLTaskMetrics objects over time
> ------------------------------------------------------------------------------------------
>
>                 Key: SPARK-11192
>                 URL: https://issues.apache.org/jira/browse/SPARK-11192
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.5.1
>         Environment: java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
> org.apache.spark/spark-sql_2.10 "1.5.1"
> Embedded, in-process spark. Have not tested on standalone or yarn clusters.
>            Reporter: Blake Livingston
>            Priority: Minor
>
> Noticed that slowly, over the course of a day or two, heap memory usage on a long running spark process increased monotonically.
> After doing a heap dump and examining in jvisualvm, saw there were over 15M org.apache.spark.sql.execution.ui.SQLTaskMetrics objects allocated, taking over 500MB.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org