incubator-zeppelin git commit: [ZEPPELIN-572] pyspark interpreter doesn't work on yarn-client

[mo...@apache.org]

Repository: incubator-zeppelin
Updated Branches:
  refs/heads/master 89c592489 -> 2ee234ac8


[ZEPPELIN-572] pyspark interpreter doesn't work on yarn-client

### What is this PR for?
Set `spark.yarn.isPython` to be `true` to distribute pyspark libraries to workers when master is `yarn-client`

### What type of PR is it?
Bug Fix

### Is there a relevant Jira issue?
[ZEPPELIN-572](https://issues.apache.org/jira/browse/ZEPPELIN-572)

### How should this be tested?
You need yarn cluster to test this PR.
Simple way to test this PR would be running below code in paragraph and see if it throws error.
```
%pyspark
print(sc.parallelize([1, 2]).count())
```
And you should be able to see that `spark.yarn.isPython` is set to `true` in **Spark UI > Environment > Spark Properties** only when you set spark.master as `yarn-client`.

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Mina Lee <mi...@nflabs.com>

Closes #605 from minahlee/ZEPPELIN-572 and squashes the following commits:

8c99de5 [Mina Lee] Set spark.yarn.isPython to be true to distribute needed pyspark libraries to workers when master is yarn-client


Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/2ee234ac
Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/2ee234ac
Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/2ee234ac

Branch: refs/heads/master
Commit: 2ee234ac8e9248d671cba25349e3065c1bc326f3
Parents: 89c5924
Author: Mina Lee <mi...@nflabs.com>
Authored: Wed Jan 6 21:23:18 2016 -0800
Committer: Lee moon soo <mo...@apache.org>
Committed: Sun Jan 10 08:16:28 2016 -0800

----------------------------------------------------------------------
 .../main/java/org/apache/zeppelin/spark/SparkInterpreter.java    | 4 ++++
 1 file changed, 4 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/2ee234ac/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
----------------------------------------------------------------------
diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
index 08eb0fa..2bf7a6b 100644
--- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
+++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
@@ -335,6 +335,10 @@ public class SparkInterpreter extends Interpreter {
       conf.set("spark.submit.pyArchives", Joiner.on(":").join(pythonLibs));
     }
 
+    // Distributes needed libraries to workers.
+    if (getProperty("master").equals("yarn-client")) {
+      conf.set("spark.yarn.isPython", "true");
+    }
 
     SparkContext sparkContext = new SparkContext(conf);
     return sparkContext;