You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sr...@apache.org on 2015/05/01 22:20:54 UTC
spark git commit: [SPARK-7281] [YARN] Add option to set AM's lib path
in client mode.
Repository: spark
Updated Branches:
refs/heads/master f53a48827 -> 7b5dd3e3c
[SPARK-7281] [YARN] Add option to set AM's lib path in client mode.
Author: Marcelo Vanzin <va...@cloudera.com>
Closes #5813 from vanzin/SPARK-7281 and squashes the following commits:
1cb6f42 [Marcelo Vanzin] [SPARK-7281] [yarn] Add option to set AM's lib path in client mode.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7b5dd3e3
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7b5dd3e3
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7b5dd3e3
Branch: refs/heads/master
Commit: 7b5dd3e3c0030087eea5a8224789352c03717c1d
Parents: f53a488
Author: Marcelo Vanzin <va...@cloudera.com>
Authored: Fri May 1 21:20:46 2015 +0100
Committer: Sean Owen <so...@cloudera.com>
Committed: Fri May 1 21:20:46 2015 +0100
----------------------------------------------------------------------
docs/running-on-yarn.md | 7 +++++++
yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala | 4 ++++
2 files changed, 11 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/7b5dd3e3/docs/running-on-yarn.md
----------------------------------------------------------------------
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index 0968fc5..b6701b6 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -190,6 +190,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes
</td>
</tr>
<tr>
+ <td><code>spark.yarn.am.extraLibraryPath</code></td>
+ <td>(none)</td>
+ <td>
+ Set a special library path to use when launching the application master in client mode.
+ </td>
+</tr>
+<tr>
<td><code>spark.yarn.maxAppAttempts</code></td>
<td>yarn.resourcemanager.am.max-attempts in YARN</td>
<td>
http://git-wip-us.apache.org/repos/asf/spark/blob/7b5dd3e3/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
----------------------------------------------------------------------
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
index b945395..31ab6b4 100644
--- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
+++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
@@ -544,6 +544,10 @@ private[spark] class Client(
}
javaOpts ++= Utils.splitCommandString(opts).map(YarnSparkHadoopUtil.escapeForShell)
}
+
+ sparkConf.getOption("spark.yarn.am.extraLibraryPath").foreach { paths =>
+ prefixEnv = Some(Utils.libraryPathEnvPrefix(Seq(paths)))
+ }
}
// For log4j configuration to reference
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org