You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Allan Douglas R. de Oliveira (JIRA)" <ji...@apache.org> on 2014/09/01 02:49:20 UTC
[jira] [Comment Edited] (SPARK-3332) Tagging is not atomic with
launching instances on EC2
[ https://issues.apache.org/jira/browse/SPARK-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14116950#comment-14116950 ]
Allan Douglas R. de Oliveira edited comment on SPARK-3332 at 9/1/14 12:49 AM:
------------------------------------------------------------------------------
[~pwendell], yes this is a good reword and I agree to revert it for now. You mentioned the two potential solutions but I think a good compromise is the one implemented by the PR which keeps the flag allowing reuse of the same security group but also allowing to match the machines by the security group in the other cases.
Perhaps more messages could be added when the flag has been used and the tagging failed.
was (Author: douglaz):
[~pwendell], yes this is a good reword and I agree to revert it for now. You mentioned the two potential solutions but I think a good compromise is the one implemented by the PR which keeps the flag allowing reuse of the same security group but also allowing to match the machines by the security group in the other cases.
> Tagging is not atomic with launching instances on EC2
> -----------------------------------------------------
>
> Key: SPARK-3332
> URL: https://issues.apache.org/jira/browse/SPARK-3332
> Project: Spark
> Issue Type: Bug
> Components: EC2
> Reporter: Allan Douglas R. de Oliveira
>
> The implementation for SPARK-2333 changed the machine membership mechanism from security groups to tags.
> This is a fundamentally flawed strategy as there aren't guarantees at all the machines will have a tag (even with a retry mechanism).
> For instance, if the script is killed after launching the instances but before setting the tags the machines will be "invisible" to a destroy command, leaving a unmanageable cluster behind.
> The initial proposal is to go back to the previous behavior for all cases but when the new flag (--security-group-prefix) is used.
> Also it's worthwhile to mention that SPARK-3180 introduced the --additional-security-group flag which is a reasonable solution to SPARK-2333 (but isn't a full replacement to all use cases of --security-group-prefix).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org