You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by li-zhihui <gi...@git.apache.org> on 2015/04/10 11:22:44 UTC
[GitHub] spark pull request: Fix: avoid printing InterruptedException when ...
GitHub user li-zhihui opened a pull request:
https://github.com/apache/spark/pull/5451
Fix: avoid printing InterruptedException when stop application in yarn mode.
Driver log print some Exception message when stop application in yarn-client mode just like
<code>Exception in thread "Yarn application state monitor" 15/04/10 10:39:04 INFO YarnClientSchedulerBackend: Shutting down all executors
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at org.apache.spark.deploy.yarn.Client.monitorApplication(Client.scala:572)
at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend$$anon$1.run(YarnClientSchedulerBackend.scala:131)</code>
It's unnecessary and puzzling.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/li-zhihui/spark si
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5451.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5451
----
commit ce5f82468f901c42c8e7d2eb59cb8136eca79c71
Author: li-zhihui <zh...@intel.com>
Date: 2015-04-10T08:32:19Z
Fix: avoid printing InterruptedException when stop application in yarn mode.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91527302
This change has the effect of making this loop un-`interrupt()`-ible since it always catches `InterruptedException`. Was that your intent? I'm not sure if that's OK. Printing the exception again doesn't stop the stack trace you cite. Is it a problem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91742251
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30062/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-92048025
https://github.com/apache/spark/pull/5479 also addresses this, has a JIRA, and is closer to a correct fix at this point. Do you mind closing this PR and reviewing that one instead?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Fix: avoid printing InterruptedException when ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91492079
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5451#discussion_r28194335
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -60,6 +60,7 @@ private[spark] class Client(
def this(clientArgs: ClientArguments) = this(clientArgs, new SparkConf())
+ private var stopMonitor = false
--- End diff --
This would have to be `@volatile`. The problem with this construction is that several callers to the `monitorApplication` method will now collide, so I'm not sure this is safe.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91552055
[Test build #30027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30027/consoleFull) for PR 5451 at commit [`ce5f824`](https://github.com/apache/spark/commit/ce5f82468f901c42c8e7d2eb59cb8136eca79c71).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91552068
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/30027/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91527327
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91742230
[Test build #30062 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30062/consoleFull) for PR 5451 at commit [`2e7ace8`](https://github.com/apache/spark/commit/2e7ace8fc9336267216c7c9b8497835fd399f202).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class FlattenedValuesSerializer(BatchedSerializer):`
* `class ExternalList(object):`
* `class ExternalListOfList(ExternalList):`
* `class GroupByKey(object):`
* `class ExternalGroupBy(ExternalMerger):`
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91642281
What Sean said. You need to make the loop stoppable, e.g., by looping on some boolean variable that is modified before the thread is interrupted.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5451#discussion_r28194336
--- Diff: yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala ---
@@ -124,27 +124,32 @@ private[spark] class YarnClientSchedulerBackend(
* If the application has exited for any reason, stop the SparkContext.
* This assumes both `client` and `appId` have already been set.
*/
- private def asyncMonitorApplication(): Thread = {
+ private def asyncMonitorApplication(): MonitorApplicationThread = {
assert(client != null && appId != null, "Application has not been submitted yet!")
- val t = new Thread {
- override def run() {
- val (state, _) = client.monitorApplication(appId, logApplicationReport = false)
- logError(s"Yarn application has already exited with state $state!")
- sc.stop()
- Thread.currentThread().interrupt()
- }
- }
+ val t = new MonitorApplicationThread()
t.setName("Yarn application state monitor")
t.setDaemon(true)
t
}
+ private class MonitorApplicationThread extends Thread{
+ override def run() {
+ val (state, _) = client.monitorApplication(appId, logApplicationReport = false)
+ logError(s"Yarn application has already exited with state $state!")
+ sc.stop()
+ Thread.currentThread().interrupt()
+ }
+ def stopMonitor() {
+ client.stopMonitorApplication()
+ }
+ }
+
/**
* Stop the scheduler. This assumes `start()` has already been called.
*/
override def stop() {
assert(client != null, "Attempted to stop this scheduler before starting it!")
- monitorThread.interrupt()
+ monitorThread.stopMonitor()
--- End diff --
The narrowest fix is just to catch `InterruptedException` here and ignore it and/or log it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by li-zhihui <gi...@git.apache.org>.
Github user li-zhihui commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91732744
@srowen I guess the exception maybe make user feel puzzled because it's unexpected, although it doesn't effect application execution.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91730076
[Test build #30062 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30062/consoleFull) for PR 5451 at commit [`2e7ace8`](https://github.com/apache/spark/commit/2e7ace8fc9336267216c7c9b8497835fd399f202).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/5451#discussion_r28194341
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -569,7 +570,7 @@ private[spark] class Client(
logApplicationReport: Boolean = true): (YarnApplicationState, FinalApplicationStatus) = {
val interval = sparkConf.getLong("spark.yarn.report.interval", 1000)
var lastState: YarnApplicationState = null
- while (true) {
+ while (!stopMonitor) {
--- End diff --
Oops, I mean you would have to catch `InterruptedException` with a `try` block around this loop, and log and/or ignore it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by li-zhihui <gi...@git.apache.org>.
Github user li-zhihui closed the pull request at:
https://github.com/apache/spark/pull/5451
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91528800
[Test build #30027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30027/consoleFull) for PR 5451 at commit [`ce5f824`](https://github.com/apache/spark/commit/ce5f82468f901c42c8e7d2eb59cb8136eca79c71).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [Yarn][minor]Fix: avoid printing InterruptedEx...
Posted by li-zhihui <gi...@git.apache.org>.
Github user li-zhihui commented on the pull request:
https://github.com/apache/spark/pull/5451#issuecomment-91730112
@srowen @vanzin Thanks for your comments.
I add a new commit as your comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org