You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2014/11/26 01:08:13 UTC
[jira] [Closed] (SPARK-3332) Tagging is not atomic with launching
instances on EC2
[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Or closed SPARK-3332.
----------------------------
Resolution: Fixed
Fix Version/s: 1.2.0
1.1.1
Assignee: Josh Rosen
Target Version/s: 1.1.1, 1.2.0 (was: 1.2.0)
> Tagging is not atomic with launching instances on EC2
> -----------------------------------------------------
>
> Key: SPARK-3332
> URL: https://issues.apache.org/jira/browse/SPARK-3332
> Project: Spark
> Issue Type: Bug
> Components: EC2
> Reporter: Allan Douglas R. de Oliveira
> Assignee: Josh Rosen
> Fix For: 1.1.1, 1.2.0
>
>
> The implementation for SPARK-2333 changed the machine membership mechanism from security groups to tags.
> This is a fundamentally flawed strategy as there aren't guarantees at all the machines will have a tag (even with a retry mechanism).
> For instance, if the script is killed after launching the instances but before setting the tags the machines will be "invisible" to a destroy command, leaving a unmanageable cluster behind.
> The initial proposal is to go back to the previous behavior for all cases but when the new flag (--security-group-prefix) is used.
> Also it's worthwhile to mention that SPARK-3180 introduced the --additional-security-group flag which is a reasonable solution to SPARK-2333 (but isn't a full replacement to all use cases of --security-group-prefix).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org