[jira] [Resolved] (SPARK-8790) BlockManager.reregister cause OOM

["Josh Rosen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Rosen resolved SPARK-8790.
-------------------------------
    Resolution: Cannot Reproduce

I'm going to resolve this issue as "Cannot Reproduce" since it's over a year old and appears to only have been reported for Spark 1.2.x. If this is still an issue in newer Spark versions then please file a new JIRA ticket. Thanks!

> BlockManager.reregister cause OOM
> ---------------------------------
>
>                 Key: SPARK-8790
>                 URL: https://issues.apache.org/jira/browse/SPARK-8790
>             Project: Spark
>          Issue Type: Bug
>          Components: Block Manager, Spark Core
>            Reporter: Patrick Liu
>         Attachments: driver.log, executor.log, webui-executor.png, webui-slow-task.png
>
>
> We run SparkSQL 1.2.1 on Yarn.
> A SQL consists of 100 tasks, most them finish in < 10s, but only 1 lasts for 16m.
> The webUI shows that the executor has running GC for 15m brfore OOM.
> The log shows that the executor first try to connect to master to report broadcast value, however the network is not available, so the executor lost heartbeat to Master. 
> Then the master require the executor to reregister. When executor are reporAllBlocks to master, the network is still not so stable, sometimes time-out.
> Finally, the executor OOM.
> Please take a look.
> Attached is the detailed log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org