You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by Sherry302 <gi...@git.apache.org> on 2016/09/28 22:11:26 UTC

[GitHub] spark pull request #15286: [SPARK-16757][Follow UP] Fix ClassCircularityErro...

GitHub user Sherry302 opened a pull request:

    https://github.com/apache/spark/pull/15286

    [SPARK-16757][Follow UP] Fix ClassCircularityError in ReplSuite tests in Maven build: use 'Class.forName' instead of 'Utils.classForName'

    ## What changes were proposed in this pull request?
    Fix ClassCircularityError in ReplSuite tests when Spark is built by Maven build.
    
    ## How was this patch tested?
    (1)
    ```
    build/mvn -DskipTests -Phadoop-2.3 -Pyarn -Phive -Phive-thriftserver -Pkinesis-asl -Pmesos clean package
    ```
    Then test:
    ```
    build/mvn -Dtest=none -DwildcardSuites=org.apache.spark.repl.ReplSuite test
    ```
    ReplSuite tests passed
    
    (2)
    Manual Tests against some Spark applications in Yarn client mode and Yarn cluster mode. Need to check if spark caller contexts are written into HDFS hdfs-audit.log and Yarn RM audit log successfully.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/Sherry302/spark SPARK-16757

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/15286.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #15286
    
----
commit 59bfa231600decfd10b29741182107b4b2c52adc
Author: Weiqing Yang <ya...@gmail.com>
Date:   2016-09-28T22:04:22Z

    [SPARK-16757][Follow UP] Fix ClassCircularityError in ReplSuite tests in Maven build: use 'Class.forName' instead of 'Utils.classForName'

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    committed this to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by Sherry302 <gi...@git.apache.org>.
Github user Sherry302 commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    Thank you very much. @tgravescs @JoshRosen 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError i...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/spark/pull/15286


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-16757][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    Can you change the JIRA number to SPARK-17710, which is the JIRA that I created for this build break?
    
    https://issues.apache.org/jira/browse/SPARK-17710


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError i...

Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/15286#discussion_r86613864
  
    --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
    @@ -2489,8 +2489,10 @@ private[spark] class CallerContext(
       def setCurrentContext(): Boolean = {
         var succeed = false
         try {
    -      val callerContext = Utils.classForName("org.apache.hadoop.ipc.CallerContext")
    -      val Builder = Utils.classForName("org.apache.hadoop.ipc.CallerContext$Builder")
    +      // scalastyle:off classforname
    --- End diff --
    
    I think it's important to document why we are not using Utils.classForName here. Otherwise somebody is going to come in the future and get confused or change it.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-16757][Follow UP] Fix ClassCircularityError in Re...

Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    please change [Follow UP]  to [HOTFIX]


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by Sherry302 <gi...@git.apache.org>.
Github user Sherry302 commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    @JoshRosen Yes. The title has been changed.
    @tgravescs Yes. Now I am creating a separate jira to investigate this more.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-16757][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    changes look fine, waiting on Jenkins.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-16757][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by tgravescs <gi...@git.apache.org>.
Github user tgravescs commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    Also please file a separate jira to investigate more.  Link it to https://issues.apache.org/jira/browse/SPARK-15857 and if needed we pull in people more familiar with the Repl classloader stuff.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError i...

Posted by weiqingy <gi...@git.apache.org>.
Github user weiqingy commented on a diff in the pull request:

    https://github.com/apache/spark/pull/15286#discussion_r86651424
  
    --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
    @@ -2489,8 +2489,10 @@ private[spark] class CallerContext(
       def setCurrentContext(): Boolean = {
         var succeed = false
         try {
    -      val callerContext = Utils.classForName("org.apache.hadoop.ipc.CallerContext")
    -      val Builder = Utils.classForName("org.apache.hadoop.ipc.CallerContext$Builder")
    +      // scalastyle:off classforname
    --- End diff --
    
    @rxin I have submitted a PR #15776 for this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66066/
    Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    **[Test build #66066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66066/consoleFull)** for PR 15286 at commit [`59bfa23`](https://github.com/apache/spark/commit/59bfa231600decfd10b29741182107b4b2c52adc).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-16757][Follow UP] Fix ClassCircularityError in Re...

Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    **[Test build #66066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66066/consoleFull)** for PR 15286 at commit [`59bfa23`](https://github.com/apache/spark/commit/59bfa231600decfd10b29741182107b4b2c52adc).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-16757][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by Sherry302 <gi...@git.apache.org>.
Github user Sherry302 commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    Yes. The title has been changed. Thanks. @tgravescs 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #15286: [SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplS...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/15286
  
    Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org