You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "liang yu (Jira)" <ji...@apache.org> on 2023/03/31 06:25:00 UTC

[jira] [Commented] (SPARK-42972) ExecutorAllocationManager cannot allocate new instances when all executors down.

    [ https://issues.apache.org/jira/browse/SPARK-42972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707143#comment-17707143 ] 

liang yu commented on SPARK-42972:
----------------------------------

When the ratio is lower than the scalingDownRatio, the client will try to kill executors, but if all executors are dead accidentally, the program will hung, because there is no executors to kill.  
Then there will be no more batch jobs to complete, and batchProcTimeCount will always be 0, the program will stuck in suspended animation.

> ExecutorAllocationManager cannot allocate new instances when all executors down.
> --------------------------------------------------------------------------------
>
>                 Key: SPARK-42972
>                 URL: https://issues.apache.org/jira/browse/SPARK-42972
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>    Affects Versions: 3.3.2
>            Reporter: Jiandan Yang 
>            Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org