[jira] [Created] (SPARK-4745) get_existing_cluster() doesn't work with additional security groups

["Alex DeBrie (JIRA)" <ji...@apache.org>]

Alex DeBrie created SPARK-4745:
----------------------------------

             Summary: get_existing_cluster() doesn't work with additional security groups
                 Key: SPARK-4745
                 URL: https://issues.apache.org/jira/browse/SPARK-4745
             Project: Spark
          Issue Type: Bug
          Components: EC2
    Affects Versions: 1.1.0
            Reporter: Alex DeBrie


The spark-ec2 script has a flag that allows you to add additional security groups to clusters when you launch. However, the get_existing_cluster() function cycles through active instances and only returns instances whose group_names == cluster_name + "-master" (or + "-slaves"), which is the group created by default.  The get_existing_cluster() function is used to login to, stop, and destroy existing clusters, among other actions.

This is a pretty simple fix for which I've already submitted a [pull request|https://github.com/apache/spark/pull/3596]. It checks if cluster_name + "-master" is in the list of groups for each active instance. This means the cluster group can be one among many groups, rather than the sole group for an instance.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org