You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Li Xian (Jira)" <ji...@apache.org> on 2021/09/24 10:52:00 UTC

[jira] [Created] (SPARK-36842) Stop task result getter properly on spark context stopping

Li Xian created SPARK-36842:
-------------------------------

             Summary: Stop task result getter properly on spark context stopping
                 Key: SPARK-36842
                 URL: https://issues.apache.org/jira/browse/SPARK-36842
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 3.3.0
            Reporter: Li Xian


org.apache.spark.scheduler.TaskSchedulerImpl#stop doesn't handle exception properly. If one component throws exceptions on stopping, the exception is thrown and TaskSchedulerImpl.stop() will not be executed completely.

For example if backend.stop() fails, then taskResultGetter.stop() won't be executed. The result is that after a couple of restart of the spark context, there will be a lot of '

task-result-getter' threads retained.

 

!image-2021-09-24-18-50-57-072.png!

!image-2021-09-24-18-51-03-837.png!

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org