You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2018/08/30 20:54:11 UTC
spark git commit: [SPARK-25235][BUILD][SHELL][FOLLOWUP] Fix repl
compile for 2.12
Repository: spark
Updated Branches:
refs/heads/master c685b5f56 -> a5fb5b62c
[SPARK-25235][BUILD][SHELL][FOLLOWUP] Fix repl compile for 2.12
## What changes were proposed in this pull request?
Error messages from https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/183/
```
[INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) spark-repl_2.12 ---
[INFO] Using zinc server for incremental compilation
[warn] Pruning sources from previous analysis, due to incompatible CompileSetup.
[info] Compiling 6 Scala sources to /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/target/scala-2.12/classes...
[error] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoopInterpreter.scala:80: overriding lazy value importableSymbolsWithRenames in class ImportHandler of type List[(this.intp.global.Symbol, this.intp.global.Name)];
[error] lazy value importableSymbolsWithRenames needs `override' modifier
[error] lazy val importableSymbolsWithRenames: List[(Symbol, Name)] = {
[error] ^
[warn] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala:53: variable addedClasspath in class ILoop is deprecated (since 2.11.0): use reset, replay or require to update class path
[warn] if (addedClasspath != "") {
[warn] ^
[warn] /home/jenkins/workspace/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala:54: variable addedClasspath in class ILoop is deprecated (since 2.11.0): use reset, replay or require to update class path
[warn] settings.classpath append addedClasspath
[warn] ^
[warn] two warnings found
[error] one error found
[error] Compile failed at Aug 29, 2018 5:28:22 PM [0.679s]
```
Readd the profile for `scala-2.12`. Using `-Pscala-2.12` will overrides `extra.source.dir` and `extra.testsource.dir` with two non-exist directories.
## How was this patch tested?
First, make sure it compiles.
```
dev/change-scala-version.sh 2.12
mvn -Pscala-2.12 -DskipTests compile install
```
Then, make a distribution to try the repl:
`./dev/make-distribution.sh --name custom-spark --tgz -Phadoop-2.7 -Phive -Pyarn -Pscala-2.12`
```
18/08/30 16:04:50 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Spark context Web UI available at http://172.16.131.140:4040
Spark context available as 'sc' (master = local[*], app id = local-1535616298812).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.4.0-SNAPSHOT
/_/
Using Scala version 2.12.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112)
Type in expressions to have them evaluated.
Type :help for more information.
scala> spark.sql("select percentile(key, 1) from values (1, 1),(2, 1) T(key, value)").show
+-------------------------------------+
|percentile(key, CAST(1 AS DOUBLE), 1)|
+-------------------------------------+
| 2.0|
+-------------------------------------+
```
Closes #22280 from sadhen/SPARK_24785_FOLLOWUP.
Authored-by: 忍冬 <re...@wacai.com>
Signed-off-by: Sean Owen <se...@databricks.com>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a5fb5b62
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a5fb5b62
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a5fb5b62
Branch: refs/heads/master
Commit: a5fb5b62c3595ce2e59e7b8e95ef5bcc825f1577
Parents: c685b5f
Author: 忍冬 <re...@wacai.com>
Authored: Thu Aug 30 15:54:07 2018 -0500
Committer: Sean Owen <se...@databricks.com>
Committed: Thu Aug 30 15:54:07 2018 -0500
----------------------------------------------------------------------
repl/pom.xml | 10 ++++++++++
1 file changed, 10 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/a5fb5b62/repl/pom.xml
----------------------------------------------------------------------
diff --git a/repl/pom.xml b/repl/pom.xml
index 553d5eb..861bbd7 100644
--- a/repl/pom.xml
+++ b/repl/pom.xml
@@ -167,4 +167,14 @@
</plugins>
</build>
+ <profiles>
+ <profile>
+ <id>scala-2.12</id>
+ <properties>
+ <extra.source.dir>scala-2.12/src/main/scala</extra.source.dir>
+ <extra.testsource.dir>scala-2.12/src/test/scala</extra.testsource.dir>
+ </properties>
+ </profile>
+ </profiles>
+
</project>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org