You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yeachan Park (Jira)" <ji...@apache.org> on 2023/03/09 14:58:00 UTC
[jira] [Updated] (SPARK-42736) Display executor remove reason statistics for clusters which don't have dynamic allocation
[ https://issues.apache.org/jira/browse/SPARK-42736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yeachan Park updated SPARK-42736:
---------------------------------
Description:
The remove reason statistics do not seem to be enabled for clusters which do not have dynamic allocation enabled. For clusters which do have dynamic allocation, we see this:
```ExecutorMonitor: Executor 2 is removed. Remove reason statistics: (gracefully decommissioned: 2, decommision unfinished: 0, driver killed: 0, unexpectedly exited: 0).```
Something similar would be helpful for clusters which don't have dynamic allocation, as executors can still be killed for whatever reason and it would be helpful to see if those killed executors were able to gracefully decommission.
was:
The remove reason statistics do not seem to be enabled for clusters which do not have dynamic allocation enabled. For clusters which do have dynamic allocation, we see this:
`ExecutorMonitor: Executor 2 is removed. Remove reason statistics: (gracefully decommissioned: 2, decommision unfinished: 0, driver killed: 0, unexpectedly exited: 0).`
Something similar would be helpful for clusters which don't have dynamic allocation, as executors can still be killed for whatever reason and it would be helpful to see if those killed executors were able to gracefully decommission.
> Display executor remove reason statistics for clusters which don't have dynamic allocation
> ------------------------------------------------------------------------------------------
>
> Key: SPARK-42736
> URL: https://issues.apache.org/jira/browse/SPARK-42736
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core
> Affects Versions: 3.3.2
> Reporter: Yeachan Park
> Priority: Minor
>
> The remove reason statistics do not seem to be enabled for clusters which do not have dynamic allocation enabled. For clusters which do have dynamic allocation, we see this:
> ```ExecutorMonitor: Executor 2 is removed. Remove reason statistics: (gracefully decommissioned: 2, decommision unfinished: 0, driver killed: 0, unexpectedly exited: 0).```
> Something similar would be helpful for clusters which don't have dynamic allocation, as executors can still be killed for whatever reason and it would be helpful to see if those killed executors were able to gracefully decommission.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org