You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by jo...@apache.org on 2014/12/04 23:16:27 UTC
spark git commit: [SPARK-4745] Fix get_existing_cluster() function
with multiple security groups
Repository: spark
Updated Branches:
refs/heads/master 8dae26f83 -> 794f3aec2
[SPARK-4745] Fix get_existing_cluster() function with multiple security groups
The current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups == cluster_name + "-master" (or "-slaves"). This fix allows for multiple security groups by checking if the cluster_name + "-master" security group is in the list of groups for a particular instance.
Author: alexdebrie <al...@gmail.com>
Closes #3596 from alexdebrie/master and squashes the following commits:
9d51232 [alexdebrie] Fix get_existing_cluster() function with multiple security groups
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/794f3aec
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/794f3aec
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/794f3aec
Branch: refs/heads/master
Commit: 794f3aec24acb578e258532ad0590554d07958ba
Parents: 8dae26f
Author: alexdebrie <al...@gmail.com>
Authored: Thu Dec 4 14:13:59 2014 -0800
Committer: Josh Rosen <jo...@databricks.com>
Committed: Thu Dec 4 14:14:39 2014 -0800
----------------------------------------------------------------------
ec2/spark_ec2.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/794f3aec/ec2/spark_ec2.py
----------------------------------------------------------------------
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py
index b83deca..5f9e484 100755
--- a/ec2/spark_ec2.py
+++ b/ec2/spark_ec2.py
@@ -504,9 +504,9 @@ def get_existing_cluster(conn, opts, cluster_name, die_on_error=True):
active = [i for i in res.instances if is_active(i)]
for inst in active:
group_names = [g.name for g in inst.groups]
- if group_names == [cluster_name + "-master"]:
+ if (cluster_name + "-master") in group_names:
master_nodes.append(inst)
- elif group_names == [cluster_name + "-slaves"]:
+ elif (cluster_name + "-slaves") in group_names:
slave_nodes.append(inst)
if any((master_nodes, slave_nodes)):
print "Found %d master(s), %d slaves" % (len(master_nodes), len(slave_nodes))
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org