You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2016/02/20 21:58:54 UTC
spark git commit: [SPARK-13414][MESOS] Allow multiple dispatchers to
be launched.
Repository: spark
Updated Branches:
refs/heads/master 6ce7c481d -> a4a081d1d
[SPARK-13414][MESOS] Allow multiple dispatchers to be launched.
## What changes were proposed in this pull request?
Users might want to start multiple mesos dispatchers, as each dispatcher can potentially be part of different roles and used for multi-tenancy.
To allow multiple Mesos dispatchers to be launched, we need to be able to specify a instance number when starting the dispatcher daemon.
## How was the this patch tested?
Manual testing
Author: Timothy Chen <tn...@gmail.com>
Closes #11281 from tnachen/multiple_cluster_dispatchers.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a4a081d1
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a4a081d1
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a4a081d1
Branch: refs/heads/master
Commit: a4a081d1df043cce6db0284ef552e5174ebb0d02
Parents: 6ce7c48
Author: Timothy Chen <tn...@gmail.com>
Authored: Sat Feb 20 12:58:47 2016 -0800
Committer: Reynold Xin <rx...@databricks.com>
Committed: Sat Feb 20 12:58:47 2016 -0800
----------------------------------------------------------------------
sbin/start-mesos-dispatcher.sh | 5 ++++-
sbin/stop-mesos-dispatcher.sh | 7 ++++++-
2 files changed, 10 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/a4a081d1/sbin/start-mesos-dispatcher.sh
----------------------------------------------------------------------
diff --git a/sbin/start-mesos-dispatcher.sh b/sbin/start-mesos-dispatcher.sh
index 4777e16..06a966d 100755
--- a/sbin/start-mesos-dispatcher.sh
+++ b/sbin/start-mesos-dispatcher.sh
@@ -37,5 +37,8 @@ if [ "$SPARK_MESOS_DISPATCHER_HOST" = "" ]; then
SPARK_MESOS_DISPATCHER_HOST=`hostname`
fi
+if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
+ SPARK_MESOS_DISPATCHER_NUM=1
+fi
-"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher 1 --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
+"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher $SPARK_MESOS_DISPATCHER_NUM --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
http://git-wip-us.apache.org/repos/asf/spark/blob/a4a081d1/sbin/stop-mesos-dispatcher.sh
----------------------------------------------------------------------
diff --git a/sbin/stop-mesos-dispatcher.sh b/sbin/stop-mesos-dispatcher.sh
index 5c0b4e0..b13e018 100755
--- a/sbin/stop-mesos-dispatcher.sh
+++ b/sbin/stop-mesos-dispatcher.sh
@@ -24,5 +24,10 @@ fi
. "${SPARK_HOME}/sbin/spark-config.sh"
-"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher 1
+if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
+ SPARK_MESOS_DISPATCHER_NUM=1
+fi
+
+"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher \
+ $SPARK_MESOS_DISPATCHER_NUM
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org