You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2016/01/03 16:24:39 UTC
[jira] [Commented] (SPARK-9111) Dumping the memory info when an
executor dies abnormally
[ https://issues.apache.org/jira/browse/SPARK-9111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080452#comment-15080452 ]
Steve Loughran commented on SPARK-9111:
---------------------------------------
the heap dump option could itself be useful; within a YARN container the launch command could be set to something like {{-XX:HeapDumpPath=<<LOGDIR>>}} then the heap dump would be automatically grabbed by the YARN Nodemanager and copied to HDFS, where it would then be cleaned up by the normal YARN history cleanup routines.
> Dumping the memory info when an executor dies abnormally
> --------------------------------------------------------
>
> Key: SPARK-9111
> URL: https://issues.apache.org/jira/browse/SPARK-9111
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Reporter: Zhang, Liye
> Priority: Minor
>
> When an executor is not normally finished, we shall give out it's memory dump info right before the JVM shutting down. So that if the executor is killed because of OOM, we can easily checkout how is the memory used and which part cause the OOM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org