You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2018/08/30 19:48:00 UTC
[jira] [Commented] (SPARK-25285) Add executor task metrics to track
the number of tasks started and of tasks successfully completed
[ https://issues.apache.org/jira/browse/SPARK-25285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597849#comment-16597849 ]
Apache Spark commented on SPARK-25285:
--------------------------------------
User 'LucaCanali' has created a pull request for this issue:
https://github.com/apache/spark/pull/22290
> Add executor task metrics to track the number of tasks started and of tasks successfully completed
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-25285
> URL: https://issues.apache.org/jira/browse/SPARK-25285
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.4.0
> Reporter: Luca Canali
> Priority: Minor
>
> The motivation for these additional metrics is to help in troubleshooting situations when tasks fail, are killed and/or restarted. Currently available metrics include executor threadpool metrics for task completed and for active tasks. The addition of threadpool tasStarted metric will allow for example to collect info on the (approximate) number of failed tasks by computing the difference thread started – (active threads + completed tasks and/or successfully completed tasks).
> The proposed metric successfulTasks is also intended for this type of troubleshooting. The difference between  successfulTasks and threadpool.completeTasks, is that the latter is a (dropwizard library) gauge taken from the threadpool, while the former is a (dropwizard) counter computed in the [[Executor]] class, when a task successfully completes, together with several other task metrics counters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org