You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by chesterxgchen <gi...@git.apache.org> on 2014/08/22 04:57:38 UTC
[GitHub] spark pull request: Spark-3177 & SPARK-3175
GitHub user chesterxgchen opened a pull request:
https://github.com/apache/spark/pull/2090
Spark-3177 & SPARK-3175
SPARK-3177 : Yarn-alpha ClientBaseSuite Unit test failed
SPARK-3175 : Branch-1.1 SBT build failed for Yarn-Alpha
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/AlpineNow/spark SPARK-3177
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2090.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2090
----
commit 1cb61ee2f1f911b02bba84436d87074d5b7a0831
Author: chesterxgchen <ch...@alpinenow.com>
Date: 2014-08-21T20:45:45Z
SPARK-3175 : Branch-1.1 SBT build failed for Yarn-Alpha
The issue is that the yarn/alpha/pom.xml using 1.1.0 instead of 1.1.1-SNAPSHOT version.
update the pom.xml to 1.1.1-SNAPSHOT (same as yarn/stable/pom.xml)
commit 61b0c8228eab53bfba9f502a121d66ddfc71f07c
Author: chesterxgchen <ch...@alpinenow.com>
Date: 2014-08-21T21:31:14Z
SPARK-3177 : Yarn-alpha ClientBaseSuite Unit test failed
The Issue is yarn-alpha and yarn has different APIs for certain class field. In this particular case, we can directly use ClientBase method (which use reflection )
val knownDefMRAppCP: Seq[String] = ClientBase.getDefaultMRApplicationClasspath.getOrElse(Seq[String]())
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53143937
QA results for PR 2090:<br>- This patch PASSES unit tests.<br>- This patch merges cleanly<br>- This patch adds no public classes<br><br>For more information see test ouptut:<br>https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19093/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by chesterxgchen <gi...@git.apache.org>.
Github user chesterxgchen closed the pull request at:
https://github.com/apache/spark/pull/2090
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53142611
QA tests have started for PR 2090. This patch merges cleanly. <br>View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19093/consoleFull
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by chesterxgchen <gi...@git.apache.org>.
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53211490
Patrick
>>The release version is incorrect in the YARN pom intentionally (it's a weird artifact of the way we publish builds).
It would nice that someone what's the reason for this.
Regarding Spark-3177, Yes, you are correct. I was only focus on the fix the error and did not pay attention on how the value is used on the test.
Let me has a better fix for this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53137965
Jenkins, test this please. Thanks LGTM pending tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53142606
The release version is incorrect in the YARN pom intentionally (it's a weird artifact of the way we publish builds). If you want to build branch-1.1 please checkout the snapshot2 committ and all of the versions will work together. Once we cut a release we'll patch it up. The other change you made causes the unit test to become invalid, so I'm not sure that's a good idea.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2090#discussion_r16608088
--- Diff: yarn/common/src/test/scala/org/apache/spark/deploy/yarn/ClientBaseSuite.scala ---
@@ -199,10 +199,7 @@ class ClientBaseSuite extends FunSuite with Matchers {
Seq[String]())(a => a.toSeq)
- val knownDefMRAppCP: Seq[String] =
- getFieldValue[String, Seq[String]](classOf[MRJobConfig],
- "DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH",
- Seq[String]())(a => a.split(","))
+ val knownDefMRAppCP: Seq[String] = ClientBase.getDefaultMRApplicationClasspath.getOrElse(Seq[String]())
--- End diff --
Does this line fit in 100 chars? (Doesn't seem to for me.)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53016829
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53088236
LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53142581
Wait actually - the current change made to the unit test causes the unit test to be entirely useless (it just compares the output of a function to itself.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by chesterxgchen <gi...@git.apache.org>.
Github user chesterxgchen commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53212454
BTW, I have to change the pom.xml in order to fix the unit test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Spark-3177 & SPARK-3175
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2090#issuecomment-53142551
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org