You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by fe...@apache.org on 2017/09/26 05:38:04 UTC
zeppelin git commit: [ZEPPELIN-2944] Cannot launch Spark interpreter
for non-local mode
Repository: zeppelin
Updated Branches:
refs/heads/master a6a8b0298 -> 184283c4c
[ZEPPELIN-2944] Cannot launch Spark interpreter for non-local mode
### What is this PR for?
When I used Spark interpreter for non-local mode, I got
```
java.lang.RuntimeException: SPARK_HOME is not specified for non-local mode
at org.apache.zeppelin.interpreter.InterpreterSetting.setupPropertiesForSparkR(InterpreterSetting.java:783)
at org.apache.zeppelin.interpreter.InterpreterSetting.getEnvFromInterpreterProperty(InterpreterSetting.java:741)
at org.apache.zeppelin.interpreter.InterpreterSetting.createInterpreterProcess(InterpreterSetting.java:712)
at org.apache.zeppelin.interpreter.ManagedInterpreterGroup.getOrCreateInterpreterProcess(ManagedInterpreterGroup.java:58)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getOrCreateInterpreterProcess(RemoteInterpreter.java:98)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:153)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:265)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:423)
at org.apache.zeppelin.scheduler.Job.run(Job.java:182)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:307)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
```
And I fixed it! 😄
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-2944](https://issues.apache.org/jira/browse/ZEPPELIN-2944)
### How should this be tested?
Build and launch Spark interpreter
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jun Kim <ta...@users.noreply.github.com>
Closes #2594 from tae-jun/patch-6 and squashes the following commits:
2b2d147 [Jun Kim] Correctly get SPARK_HOME
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/184283c4
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/184283c4
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/184283c4
Branch: refs/heads/master
Commit: 184283c4cf204f4be51c6260a50ccb7581461bfc
Parents: a6a8b02
Author: Jun Kim <ta...@users.noreply.github.com>
Authored: Wed Sep 20 22:20:47 2017 +0900
Committer: Felix Cheung <fe...@apache.org>
Committed: Mon Sep 25 22:38:01 2017 -0700
----------------------------------------------------------------------
.../java/org/apache/zeppelin/interpreter/InterpreterSetting.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/184283c4/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java
index 9a453d8..3a808c5 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java
@@ -738,7 +738,7 @@ public class InterpreterSetting {
}
setupPropertiesForPySpark(sparkProperties);
- setupPropertiesForSparkR(sparkProperties, javaProperties.getProperty("SPARK_HOME"));
+ setupPropertiesForSparkR(sparkProperties, System.getenv("SPARK_HOME"));
if (isYarnMode() && getDeployMode().equals("cluster")) {
env.put("SPARK_YARN_CLUSTER", "true");
}