You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by chesterxgchen <gi...@git.apache.org> on 2014/08/25 03:03:57 UTC

[GitHub] spark pull request: SPARK-3177 : Yarn-alpha ClientBaseSuite Unit t...

GitHub user chesterxgchen opened a pull request:

    https://github.com/apache/spark/pull/2111

    SPARK-3177 : Yarn-alpha ClientBaseSuite Unit test failed

    This second try to fix the Yarn-Alpha unit test failure due to Yarn API changes. 
    
    I have to include SPARK-3175 (pom.xml) changes in order to run the test. Otherwise, the SBT won't be able to start the project. 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/AlpineNow/spark SPARK-3177

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/2111.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2111
    
----
commit c594232b98bbe9214e27b0a9d15e64aad7641c83
Author: chesterxgchen <ch...@alpinenow.com>
Date:   2014-08-21T20:45:45Z

    SPARK-3175 : Branch-1.1 SBT build failed for Yarn-Alpha
    
    The issue is that the yarn/alpha/pom.xml using 1.1.0 instead of 1.1.1-SNAPSHOT version.
    
    update the pom.xml to 1.1.1-SNAPSHOT (same as yarn/stable/pom.xml)

commit 30a0ac910808556f1e3477fd077b144eb6f8d9f1
Author: chesterxgchen <ch...@alpinenow.com>
Date:   2014-08-21T21:31:14Z

    SPARK-3177 : Yarn-alpha ClientBaseSuite Unit test failed
    
     The Issue is yarn-alpha and yarn has different APIs for certain class field. In this particular case, we can directly use ClientBase method (which use reflection )
    
    Add a new method
    
      def getFieldValue2[A: ClassTag, A1: ClassTag, B](clazz: Class[_], field: String,
                                                          defaults: => B)
                                  (mapTo:  A => B)(mapTo1: A1 => B) : B =
        Try(clazz.getField(field)).map(_.get(null)).map {
          case v: A => mapTo(v)
          case v1: A1 => mapTo1(v1)
          case _ => defaults
        }.toOption.getOrElse(defaults)
    
    to handle the cases where the field type can be either type A or A1. In this new method the type A or A1 is pattern matched and corresponding mapTo function (mapTo or mapTo1) is used.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: SPARK-3177 : Yarn-alpha ClientBaseSuite Unit t...

Posted by tgravescs <gi...@git.apache.org>.

Github user tgravescs commented on the pull request:

    https://github.com/apache/spark/pull/2111#issuecomment-53462912
  
    We should check to see if i changed in any of the other minor versions before hadoop2.2. Otherwise it will work for you but not others.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: SPARK-3177 : Yarn-alpha ClientBaseSuite Unit t...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2111#issuecomment-53264010
  
    QA tests have started for PR 2111. This patch merges cleanly. <br>View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19138/consoleFull


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: SPARK-3177 : Yarn-alpha ClientBaseSuite Unit t...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/2111#issuecomment-53274969
  
    QA results for PR 2111:<br>- This patch PASSES unit tests.<br>- This patch merges cleanly<br>- This patch adds no public classes<br><br>For more information see test ouptut:<br>https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19138/consoleFull


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org