You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by co...@apache.org on 2016/09/01 07:21:51 UTC
zeppelin git commit: ZEPPELIN-1384. Spark interpreter binary
compatibility to scala 2.10 / 2.11 broken
Repository: zeppelin
Updated Branches:
refs/heads/master df2e77de9 -> d93fb7361
ZEPPELIN-1384. Spark interpreter binary compatibility to scala 2.10 / 2.11 broken
### What is this PR for?
As described in ZEPPELIN-1384, we may hit the following error when running zeppelin on spark 1.6 if we build zeppelin this way
```
dev/change_scala_version.sh 2.11
mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0 -Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package
```
```
java.lang.NoSuchMethodError: scala.runtime.VolatileByteRef.create(B)Lscala/runtime/VolatileByteRef;
at scala.xml.MetaData$.iterate$1(MetaData.scala:39)
at scala.xml.MetaData$.normalize(MetaData.scala:45)
at scala.xml.Elem.<init>(Elem.scala:99)
at org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:57)
at org.apache.spark.ui.jobs.StagePage$$anonfun$26.apply(StagePage.scala:55)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at org.apache.spark.ui.jobs.StagePage.<init>(StagePage.scala:55)
at org.apache.spark.ui.jobs.StagesTab.<init>(StagesTab.scala:34)
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:57)
at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:195)
at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:146)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:473)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_1(SparkInterpreter.java:440)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:354)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:137)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:743)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
```
The root cause is that scala-xml is removed from scala 2.11 to a separate library, so here we have class conflict of scala-xml api. In this PR, I make the scope of scala-xml to be provided and also make the scope of scala-library to be provided although it will be override in `ZEPPELIN_HOME/spark/pom.xml`
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1384
### How should this be tested?
Tested manually.
Use the following command to build zeppelin and then run it on spark-1.6.2
```
dev/change_scala_version.sh 2.11
mvn -DskipTests -Drat.skip=true -Pscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0 -Phadoop-2.6 -Pyarn -Ppyspark -Psparkr clean package
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zj...@apache.org>
Closes #1377 from zjffdu/ZEPPELIN-1384 and squashes the following commits:
eb07535 [Jeff Zhang] ZEPPELIN-1384. Spark interpreter binary compatibility to scala 2.10 / 2.11 broken
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/d93fb736
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/d93fb736
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/d93fb736
Branch: refs/heads/master
Commit: d93fb7361d73d1cdbf6a551b3892d4dd8c69656d
Parents: df2e77d
Author: Jeff Zhang <zj...@apache.org>
Authored: Mon Aug 29 09:58:29 2016 +0800
Committer: Damien CORNEAU <co...@gmail.com>
Committed: Thu Sep 1 16:21:45 2016 +0900
----------------------------------------------------------------------
zeppelin-display/pom.xml | 2 ++
1 file changed, 2 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/d93fb736/zeppelin-display/pom.xml
----------------------------------------------------------------------
diff --git a/zeppelin-display/pom.xml b/zeppelin-display/pom.xml
index c30b061..7631d4f 100644
--- a/zeppelin-display/pom.xml
+++ b/zeppelin-display/pom.xml
@@ -81,6 +81,7 @@
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
+ <scope>provided</scope>
</dependency>
<dependency>
@@ -99,6 +100,7 @@
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-xml_${scala.binary.version}</artifactId>
<version>1.0.2</version>
+ <scope>provided</scope>
</dependency>
</dependencies>
</profile>