You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by piaozhexiu <gi...@git.apache.org> on 2015/02/26 00:05:02 UTC
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
GitHub user piaozhexiu opened a pull request:
https://github.com/apache/spark/pull/4773
[SPARK-6018] [YARN] NoSuchMethodError in Spark app is swallowed by YARN AM
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/piaozhexiu/spark SPARK-6018
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4773.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4773
----
commit eb5750d224132acdd7b934ada2d9c05518189e64
Author: Cheolsoo Park <ch...@netflix.com>
Date: 2015-02-25T23:00:21Z
NoSuchMethodError in Spark app is swallowed by YARN AM
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/4773#discussion_r25389917
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala ---
@@ -483,14 +483,14 @@ private[spark] class ApplicationMaster(
} catch {
case e: InvocationTargetException =>
e.getCause match {
- case _: InterruptedException =>
+ case e: InterruptedException =>
// Reporter thread can interrupt to stop user class
- case e: Exception =>
+ case _ =>
--- End diff --
So, the problem is that this was not catching `Error`, but just `Exception`, right?
I think this generates a compiler warning, so probably should be `case _: Throwable =>`. And you can avoid the duplicate `getCause` calls by giving this variable a name instead of using the placeholder.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4773#discussion_r25391907
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala ---
@@ -485,7 +485,7 @@ private[spark] class ApplicationMaster(
e.getCause match {
case _: InterruptedException =>
// Reporter thread can interrupt to stop user class
- case e: Exception =>
+ case e: Throwable =>
--- End diff --
`e` still shadows the `e: InvocationTargetException` above. I know it was already that way in the code and it works as intended this way. However while we're here I wonder if it's worth calling this variable `cause`? then it's obvious that `cause` is being rethrown.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by piaozhexiu <gi...@git.apache.org>.
Github user piaozhexiu commented on a diff in the pull request:
https://github.com/apache/spark/pull/4773#discussion_r25392875
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala ---
@@ -485,7 +485,7 @@ private[spark] class ApplicationMaster(
e.getCause match {
case _: InterruptedException =>
// Reporter thread can interrupt to stop user class
- case e: Exception =>
+ case e: Throwable =>
--- End diff --
Sure. That is even better. Fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76105349
[Test build #27967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27967/consoleFull) for PR 4773 at commit [`2a919d5`](https://github.com/apache/spark/commit/2a919d5544f0c4d39e931e57bd84ae6c57514109).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76105354
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27967/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76105552
LGTM. Let me leave it open for a day at least for other thoughts.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by andrewor14 <gi...@git.apache.org>.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76282207
Ah, good catch. For a second I thought we were catching throwable here, but it seems that the throwable is part of the cause, not what we're catching. LGTM I'm merging this into master 1.3.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76089555
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by piaozhexiu <gi...@git.apache.org>.
Github user piaozhexiu commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76085929
Thank you Marcelo and Sean. Incorporated your comments in the new commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4773
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76091021
[Test build #27967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27967/consoleFull) for PR 4773 at commit [`2a919d5`](https://github.com/apache/spark/commit/2a919d5544f0c4d39e931e57bd84ae6c57514109).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4773#discussion_r25390854
--- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala ---
@@ -483,14 +483,14 @@ private[spark] class ApplicationMaster(
} catch {
case e: InvocationTargetException =>
e.getCause match {
- case _: InterruptedException =>
+ case e: InterruptedException =>
--- End diff --
Nit: reusing the name `e` is ambiguous but also unnecessary; this can be `_` too? (Yes the second clause will generate a warning without `: Throwable`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6018] [YARN] NoSuchMethodError in Spark...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4773#issuecomment-76081377
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org