git commit: SPARK-1156: allow user to login into a cluster without slaves

[ma...@apache.org]

Repository: spark
Updated Branches:
  refs/heads/master cda381f88 -> 3eb009f36


SPARK-1156: allow user to login into a cluster without slaves

Reported in https://spark-project.atlassian.net/browse/SPARK-1156

The current spark-ec2 script doesn't allow user to login to a cluster without slaves. One of the issues brought by this behaviour is that when all the worker died, the user cannot even login to the cluster for debugging, etc.

Author: CodingCat <zh...@gmail.com>

Closes #58 from CodingCat/SPARK-1156 and squashes the following commits:

104af07 [CodingCat] output ERROR to stderr
9a71769 [CodingCat] do not allow user to start 0-slave cluster
24a7c79 [CodingCat] allow user to login into a cluster without slaves


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3eb009f3
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/3eb009f3
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/3eb009f3

Branch: refs/heads/master
Commit: 3eb009f362993dbe43028419c2d48011111a200d
Parents: cda381f
Author: CodingCat <zh...@gmail.com>
Authored: Wed Mar 5 21:47:34 2014 -0800
Committer: Matei Zaharia <ma...@databricks.com>
Committed: Wed Mar 5 21:47:34 2014 -0800

----------------------------------------------------------------------
 ec2/spark_ec2.py | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/3eb009f3/ec2/spark_ec2.py
----------------------------------------------------------------------
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py
index 25e8538..d8840c9 100755
--- a/ec2/spark_ec2.py
+++ b/ec2/spark_ec2.py
@@ -398,15 +398,13 @@ def get_existing_cluster(conn, opts, cluster_name, die_on_error=True):
   if any((master_nodes, slave_nodes)):
     print ("Found %d master(s), %d slaves" %
            (len(master_nodes), len(slave_nodes)))
-  if (master_nodes != [] and slave_nodes != []) or not die_on_error:
+  if master_nodes != [] or not die_on_error:
     return (master_nodes, slave_nodes)
   else:
     if master_nodes == [] and slave_nodes != []:
-      print "ERROR: Could not find master in group " + cluster_name + "-master"
-    elif master_nodes != [] and slave_nodes == []:
-      print "ERROR: Could not find slaves in group " + cluster_name + "-slaves"
+      print >> sys.stderr, "ERROR: Could not find master in group " + cluster_name + "-master"
     else:
-      print "ERROR: Could not find any existing cluster"
+      print >> sys.stderr, "ERROR: Could not find any existing cluster"
     sys.exit(1)
 
 
@@ -680,6 +678,9 @@ def real_main():
     opts.zone = random.choice(conn.get_all_zones()).name
 
   if action == "launch":
+    if opts.slaves <= 0:
+      print >> sys.stderr, "ERROR: You have to start at least 1 slave"
+      sys.exit(1)
     if opts.resume:
       (master_nodes, slave_nodes) = get_existing_cluster(
           conn, opts, cluster_name)