You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2016/01/13 12:51:39 UTC
[jira] [Commented] (SPARK-12803) Consider adding ability to profile
specific instances of executors in spark
[ https://issues.apache.org/jira/browse/SPARK-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096058#comment-15096058 ]
Sean Owen commented on SPARK-12803:
-----------------------------------
What do you mean here by profiling -- generating heap dumps or hprof output? or connecting with a profiler? I don't see a problem with configuring all executors to allow profiling at the time you need to do it.
> Consider adding ability to profile specific instances of executors in spark
> ---------------------------------------------------------------------------
>
> Key: SPARK-12803
> URL: https://issues.apache.org/jira/browse/SPARK-12803
> Project: Spark
> Issue Type: Improvement
> Components: Java API
> Reporter: Rajesh Balamohan
>
> It would be useful to profile specific instances of executors as opposed to adding profiler details to all executors via "spark.executor.extraJavaOptions".
> Setting the number of executors to just 1 and profiling wouldn't be much useful (in some cases, most of the time with single executor mode would be spent in terms of reading data from remote node). At the same time, setting profiling option to all executors could just create too many number of snapshots; making it harder to analyze.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org