You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by an...@apache.org on 2015/06/30 02:28:31 UTC
spark git commit: [SPARK-8019] [SPARKR] Support SparkR spawning
worker R processes with a command other then Rscript
Repository: spark
Updated Branches:
refs/heads/master d7f796da4 -> 4a9e03fa8
[SPARK-8019] [SPARKR] Support SparkR spawning worker R processes with a command other then Rscript
This is a simple change to add a new environment variable
"spark.sparkr.r.command" that specifies the command that SparkR will
use when creating an R engine process. If this is not specified,
"Rscript" will be used by default.
I did not add any documentation, since I couldn't find any place where
environment variables (such as "spark.sparkr.use.daemon") are
documented.
I also did not add a unit test. The only test that would work
generally would be one starting SparkR with
sparkR.init(sparkEnvir=list(spark.sparkr.r.command="Rscript")), just
using the default value. I think that this is a low-risk change.
Likely committers: shivaram
Author: Michael Sannella x268 <ms...@tibco.com>
Closes #6557 from msannell/altR and squashes the following commits:
7eac142 [Michael Sannella x268] add spark.sparkr.r.command config parameter
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4a9e03fa
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/4a9e03fa
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/4a9e03fa
Branch: refs/heads/master
Commit: 4a9e03fa850af9e4ee56d011671faa04fb601170
Parents: d7f796d
Author: Michael Sannella x268 <ms...@tibco.com>
Authored: Mon Jun 29 17:28:28 2015 -0700
Committer: Andrew Or <an...@databricks.com>
Committed: Mon Jun 29 17:28:28 2015 -0700
----------------------------------------------------------------------
core/src/main/scala/org/apache/spark/api/r/RRDD.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/4a9e03fa/core/src/main/scala/org/apache/spark/api/r/RRDD.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/api/r/RRDD.scala b/core/src/main/scala/org/apache/spark/api/r/RRDD.scala
index 4dfa732..5246765 100644
--- a/core/src/main/scala/org/apache/spark/api/r/RRDD.scala
+++ b/core/src/main/scala/org/apache/spark/api/r/RRDD.scala
@@ -391,7 +391,7 @@ private[r] object RRDD {
}
private def createRProcess(rLibDir: String, port: Int, script: String): BufferedStreamThread = {
- val rCommand = "Rscript"
+ val rCommand = SparkEnv.get.conf.get("spark.sparkr.r.command", "Rscript")
val rOptions = "--vanilla"
val rExecScript = rLibDir + "/SparkR/worker/" + script
val pb = new ProcessBuilder(List(rCommand, rOptions, rExecScript))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org