You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by wchswchs <gi...@git.apache.org> on 2014/02/21 16:11:28 UTC
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
GitHub user wchswchs opened a pull request:
https://github.com/apache/incubator-spark/pull/628
add threadPool shutdown hook when kill task
is it right？
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wchswchs/incubator-spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-spark/pull/628.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #628
----
commit a8441c8da723afcba1837a1fe6eb884ef19288d8
Author: jobs wang <wc...@gmail.com>
Date: 2014-02-21T14:43:02Z
add threadPool shutdown hook when kill task
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by mridulm <gi...@git.apache.org>.
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/628#discussion_r9949391
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -141,6 +141,7 @@ private[spark] class Executor(
val tr = runningTasks.get(taskId)
if (tr != null) {
tr.kill()
+ threadPool.shutdownNow()
}
--- End diff --
This would end up killing not just the requested task - but all tasks in the thread pool - not what is required.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by wchswchs <gi...@git.apache.org>.
Github user wchswchs commented on the pull request:
https://github.com/apache/incubator-spark/pull/628#issuecomment-35863734
ok，i have closed it！
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/incubator-spark/pull/628#issuecomment-35738561
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by mridulm <gi...@git.apache.org>.
Github user mridulm commented on the pull request:
https://github.com/apache/incubator-spark/pull/628#issuecomment-35744380
remove will remove the task from the queue assuming it is not yet started : but that should not be a problem - since task.kill sets flags such that it will not execute for long even if it was not removed.
But once the task starts executing, unfortunately, there is no guaranteed way to kill it.
You can try interrupting it - assuming the task responds to interruptions (and does not gobble up InterruptedException : particularly since it is executing user code).
But other than that, I can think of a good solution ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by wchswchs <gi...@git.apache.org>.
Github user wchswchs closed the pull request at:
https://github.com/apache/incubator-spark/pull/628
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by mateiz <gi...@git.apache.org>.
Github user mateiz commented on the pull request:
https://github.com/apache/incubator-spark/pull/628#issuecomment-35863650
Given this, can you close the pull request? Or do you plan to try interrupt? That may also not fix the issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by wchswchs <gi...@git.apache.org>.
Github user wchswchs commented on a diff in the pull request:
https://github.com/apache/incubator-spark/pull/628#discussion_r9950139
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -141,6 +141,7 @@ private[spark] class Executor(
val tr = runningTasks.get(taskId)
if (tr != null) {
tr.kill()
+ threadPool.shutdownNow()
}
--- End diff --
just call threadPool.remove(tr) to remove the task?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by wchswchs <gi...@git.apache.org>.
Github user wchswchs commented on the pull request:
https://github.com/apache/incubator-spark/pull/628#issuecomment-35744706
ok ,i know, thx
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---
[GitHub] incubator-spark pull request: add threadPool shutdown hook when ki...
Posted by wchswchs <gi...@git.apache.org>.
Github user wchswchs commented on the pull request:
https://github.com/apache/incubator-spark/pull/628#issuecomment-35743010
just use threadPool.remove(tr) ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastructure@apache.org or file a JIRA ticket with INFRA.
---