You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by fe...@apache.org on 2016/05/26 04:04:51 UTC

incubator-zeppelin git commit: Make zeppelin work with CDH5.7.0

Repository: incubator-zeppelin
Updated Branches:
  refs/heads/master aff2755eb -> 78c7b5567


Make zeppelin work with CDH5.7.0

### What is this PR for?
The downloadable zeppelin install wasn;t working with CDH 5.7.0. I had to change a few things to make it work nicely together.

### What type of PR is it?
Bug Fix

### Todos

### What is the Jira issue?
https://www.mail-archive.com/userszeppelin.incubator.apache.org/msg03471.html

### How should this be tested?
Install local cdh5.7.0 (see https://github.com/krisgeus/ansible_local_cdh_hadoop)
build zeppeling (with this patch) with the following options:
mvn clean package -DskipTests -Pspark-1.6 -Phadoop-2.6 -Dspark.version=1.6.0-cdh5.7.0 -Dhadoop.version=2.6.0-cdh5.7.0 -Pvendor-repo

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? Didn't verify
* Does this needs documentation? Don't think so

Author: Kris Geusebroek <kg...@KgMBP2015.local>

Closes #868 from krisgeus/master and squashes the following commits:

e33d520 [Kris Geusebroek] Revert "Don't use extra driver classpath option since I use SPARK_CLASSPATH in spark-env.sh and using both is not supported"
488cce6 [Kris Geusebroek] Added logging and comments to clarify reason not throwing exception
146b524 [Kris Geusebroek] Don't use extra driver classpath option since I use SPARK_CLASSPATH in spark-env.sh and using both is not supported
24ea584 [Kris Geusebroek] method classServerUri not available in cdh5.7.0 Spark version. Only set config if variable is filled
50717dd [Kris Geusebroek] Use slf4j instead of parquet bundled one since parquet doesn't bundle it anymore


Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/78c7b556
Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/78c7b556
Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/78c7b556

Branch: refs/heads/master
Commit: 78c7b5567e7fb4985cecf147c39033c554dfc208
Parents: aff2755
Author: Kris Geusebroek <kg...@KgMBP2015.local>
Authored: Fri May 13 08:38:11 2016 +0200
Committer: Felix Cheung <fe...@apache.org>
Committed: Wed May 25 21:04:49 2016 -0700

----------------------------------------------------------------------
 .../org/apache/zeppelin/spark/SparkInterpreter.java     | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/78c7b556/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
----------------------------------------------------------------------
diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
index 7d134ee..60613d3 100644
--- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
+++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
@@ -276,15 +276,21 @@ public class SparkInterpreter extends Interpreter {
         classServerUri = (String) classServer.invoke(interpreter.intp());
       } catch (NoSuchMethodException | SecurityException | IllegalAccessException
           | IllegalArgumentException | InvocationTargetException e) {
-        throw new InterpreterException(e);
+        // continue instead of: throw new InterpreterException(e);
+        // Newer Spark versions (like the patched CDH5.7.0 one) don't contain this method
+        logger.warn(String.format("Spark method classServerUri not available due to: [%s]", 
+          e.getMessage()));
       }
     }
 
     SparkConf conf =
         new SparkConf()
             .setMaster(getProperty("master"))
-            .setAppName(getProperty("spark.app.name"))
-            .set("spark.repl.class.uri", classServerUri);
+            .setAppName(getProperty("spark.app.name"));
+
+    if (classServerUri != null) {
+      conf.set("spark.repl.class.uri", classServerUri);
+    }
 
     if (jars.length > 0) {
       conf.setJars(jars);