You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Zhenhua Wang (Jira)" <ji...@apache.org> on 2020/08/30 04:21:00 UTC

[jira] [Created] (SPARK-32738) thread safe endpoints may hang due to fatal exception

Zhenhua Wang created SPARK-32738:
------------------------------------

             Summary: thread safe endpoints may hang due to fatal exception
                 Key: SPARK-32738
                 URL: https://issues.apache.org/jira/browse/SPARK-32738
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 3.0.0, 2.4.6, 2.3.4
            Reporter: Zhenhua Wang


Processing for `ThreadSafeRpcEndpoint` is controlled by 'numActiveThreads' in `Inbox`. Now if any fatal exception happens during `Inbox.process`, 'numActiveThreads' is not reduced. Then other threads can not process messages in that inbox, which causes the endpoint to hang.

This problem is more serious in previous Spark 2.x versions since the driver, executor and block manager endpoints are all thread safe endpoints.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org