You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by mo...@apache.org on 2016/02/23 04:11:19 UTC
incubator-zeppelin git commit: ZEPPELIN-631 Allow user to configure
spark.yarn.dist.files
Repository: incubator-zeppelin
Updated Branches:
refs/heads/master d2f9e6475 -> 8cdf71522
ZEPPELIN-631 Allow user to configure spark.yarn.dist.files
### What is this PR for?
This enables to append values into spark.yarn.dist.files
### What type of PR is it?
Bug Fix
### Todos
* [x] - Fix the logic to update values of spark.yarn.dis.files
### Is there a relevant Jira issue?
ZEPPELIN-631
### How should this be tested?
TBD
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jongyoul Lee <jo...@gmail.com>
Closes #735 from jongyoul/ZEPPELIN-631 and squashes the following commits:
52f97ff [Jongyoul Lee] ZEPPELIN-631 Allow user to configure spark.yarn.dist.files - Enabled to append values into spark.yarn.dist.files
Project: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/commit/8cdf7152
Tree: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/tree/8cdf7152
Diff: http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/diff/8cdf7152
Branch: refs/heads/master
Commit: 8cdf715220cdb3034c4992d38cac96f11f9988dd
Parents: d2f9e64
Author: Jongyoul Lee <jo...@gmail.com>
Authored: Fri Feb 19 12:18:25 2016 +0900
Committer: Lee moon soo <mo...@apache.org>
Committed: Mon Feb 22 19:14:43 2016 -0800
----------------------------------------------------------------------
.../main/java/org/apache/zeppelin/spark/SparkInterpreter.java | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-zeppelin/blob/8cdf7152/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
----------------------------------------------------------------------
diff --git a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
index be891d8..a905fb7 100644
--- a/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
+++ b/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
@@ -327,7 +327,12 @@ public class SparkInterpreter extends Interpreter {
}
pythonLibUris.trimToSize();
if (pythonLibs.length == pythonLibUris.size()) {
- conf.set("spark.yarn.dist.files", Joiner.on(",").join(pythonLibUris));
+ try {
+ String confValue = conf.get("spark.yarn.dist.files");
+ conf.set("spark.yarn.dist.files", confValue + "," + Joiner.on(",").join(pythonLibUris));
+ } catch (NoSuchElementException e) {
+ conf.set("spark.yarn.dist.files", Joiner.on(",").join(pythonLibUris));
+ }
if (!useSparkSubmit()) {
conf.set("spark.files", conf.get("spark.yarn.dist.files"));
}