You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by sadhen <gi...@git.apache.org> on 2018/08/30 07:29:05 UTC

[GitHub] spark pull request #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl...

GitHub user sadhen opened a pull request:

    https://github.com/apache/spark/pull/22280

    [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compile for 2.12

    ## What changes were proposed in this pull request?
    
    Error messages from https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/183/
    
    ```
    [INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) @ spark-repl_2.12 ---
    [INFO] Using zinc server for incremental compilation
    [warn] Pruning sources from previous analysis, due to incompatible CompileSetup.
    [info] Compiling 6 Scala sources to /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/target/scala-2.12/classes...
    [error] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala:80: overriding lazy value importableSymbolsWithRenames in class ImportHandler of type List[(this.intp.global.Symbol, this.intp.global.Name)];
    [error]  lazy value importableSymbolsWithRenames needs `override' modifier
    [error]       lazy val importableSymbolsWithRenames: List[(Symbol, Name)] = {
    [error]                ^
    [warn] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala:53: variable addedClasspath in class ILoop is deprecated (since 2.11.0): use reset, replay or require to update class path
    [warn]       if (addedClasspath != "") {
    [warn]           ^
    [warn] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala:54: variable addedClasspath in class ILoop is deprecated (since 2.11.0): use reset, replay or require to update class path
    [warn]         settings.classpath append addedClasspath
    [warn]                                   ^
    [warn] two warnings found
    [error] one error found
    [error] Compile failed at Aug 29, 2018 5:28:22 PM [0.679s]
    ```
    
    Readd the profile for `scala-2.12`. Using `-Pscala-2.12` will overrides `extra.source.dir` and `extra.testsource.dir` with two non-exist directories.
    
    ## How was this patch tested?
    ```
    dev/change-scala-version.sh 2.12
    
    mvn -Pscala-2.12 -DskipTests compile install
    ```
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sadhen/spark SPARK_24785_FOLLOWUP

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/22280.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #22280
    
----
commit de4f543a98834b519a4ac0b5f638e176e1c06967
Author: 忍冬 <re...@...>
Date:   2018-08-30T07:21:46Z

    fix repl compile for 2.12

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    **[Test build #4307 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4307/testReport)** for PR 22280 at commit [`de4f543`](https://github.com/apache/spark/commit/de4f543a98834b519a4ac0b5f638e176e1c06967).
     * This patch **fails Spark unit tests**.
     * This patch merges cleanly.
     * This patch adds no public classes.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by sadhen <gi...@git.apache.org>.

Github user sadhen commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    This PR intends to fix the build error by mvn. 
    
    I have no idea about `scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala` .
    
    Maybe we should spare some time to eliminate the code under `scala-2.11`.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    **[Test build #4307 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4307/testReport)** for PR 22280 at commit [`de4f543`](https://github.com/apache/spark/commit/de4f543a98834b519a4ac0b5f638e176e1c06967).


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by dbtsai <gi...@git.apache.org>.

Github user dbtsai commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    @sadhen Thanks for adding back the 2.12 profile which I removed mistakenly in the code unification work. @srowen I agree we should tweak the pom file so we only add the extra source dirs when 2.11 is building. Adding empty dirs is very confusing. 



---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    Hm, why would this help? there is no scala-2.12 source dir anymore. Does this method exist in 2.11 and 2.12? then we can just add the override I guess.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    I'm going to merge this as a partial revert of the other commit, as the dummy dir this reintroduces doesn't matter. We can tweak it more later, and I want to unblock finishing 2.12 integration.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    Merged build finished. Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    Jenkins add to whitelist


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by AmplabJenkins <gi...@git.apache.org>.

Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    Test PASSed.
    Refer to this link for build results (access rights to CI server needed): 
    https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95478/
    Test PASSed.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark issue #22280: [SPARK-25235] [Build] [SHELL] [FOLLOWUP] Fix repl compil...

Posted by sadhen <gi...@git.apache.org>.

Github user sadhen commented on the issue:

    https://github.com/apache/spark/pull/22280
  
    First, `scala-2.12` and `scala-2.11` is the conventions from sbt. `scala-2.11` won't be compiled if we are using 2.12.
    
    Actually, only `<extra.source.dir>scala-2.11/src/main/scala</extra.source.dir>` is effective in the original `pom.xml`. And `<extra.testsource.dir>scala-2.11/src/test/scala</extra.testsource.dir>` is a non-exist dir.
    
    Add a `scala-2.12` profile, override `extra.source.dir` and `extra.testsource.dir` with two **standard** dirs.
    
    We may not care about whether the dirs are empty, we care about the consistent behavior with sbt.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org