You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@libcloud.apache.org by "David LaBissoniere (JIRA)" <ji...@apache.org> on 2011/07/21 19:04:58 UTC
[dev] [jira] [Created] (LIBCLOUD-104) EC2 driver automatic tag creation
can fail
EC2 driver automatic tag creation can fail
------------------------------------------
Key: LIBCLOUD-104
URL: https://issues.apache.org/jira/browse/LIBCLOUD-104
Project: Libcloud
Issue Type: Bug
Components: Compute
Affects Versions: 0.5.2
Reporter: David LaBissoniere
Priority: Minor
The EC2 driver automatically creates tags for new nodes in create_node(). Rarely, this can fail with an InvalidInstanceID.NotFound error from EC2. This error bubbles up to the create_node caller and prevents the new node information from being returned.
I've seen similar issues before on EC2 when doing a instance query right after a launch. The new node isn't always included in the result, perhaps due to some caching or data propagation issue. I suspect the same thing is happening in the tag creation API. Perhaps the ex_create_tags calls could be wrapped in a try/except with this error logged and ignored.
Here is a stack trace:
File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
compute/drivers/ec2.py", line 862, in create_node self.ex_create_tags(node=node, tags=tags)
File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
compute/drivers/ec2.py", line 672, in ex_create_tags
params=params.copy()).object File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
common/base.py", line 397, in request
response = self.responseCls(self.connection.getresponse())
File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
common/base.py", line 48, in __init__ raise Exception(self.parse_error())
Exception: InvalidInstanceID.NotFound: The instance ID 'i-55c02c12' does not exist
Note that the instance was in fact started.
I also captured the libcloud request/response log and can provide that if needed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-104) EC2 driver automatic tag creation
can fail
Posted by "Tomaz Muraus (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139869#comment-13139869 ]
Tomaz Muraus commented on LIBCLOUD-104:
---------------------------------------
I have committed a partial "fix" in r1195261 (http://svn.apache.org/viewvc?view=revision&revision=1195261).
Now tag creation failure inside create_node is considered non fatal and is ignored.
> EC2 driver automatic tag creation can fail
> ------------------------------------------
>
> Key: LIBCLOUD-104
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-104
> Project: Libcloud
> Issue Type: Bug
> Components: Compute
> Affects Versions: 0.5.2
> Reporter: David LaBissoniere
>
> The EC2 driver automatically creates tags for new nodes in create_node(). Rarely, this can fail with an InvalidInstanceID.NotFound error from EC2. This error bubbles up to the create_node caller and prevents the new node information from being returned.
> I've seen similar issues before on EC2 when doing a instance query right after a launch. The new node isn't always included in the result, perhaps due to some caching or data propagation issue. I suspect the same thing is happening in the tag creation API. Perhaps the ex_create_tags calls could be wrapped in a try/except with this error logged and ignored.
> Here is a stack trace:
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 862, in create_node self.ex_create_tags(node=node, tags=tags)
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 672, in ex_create_tags
> params=params.copy()).object File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 397, in request
> response = self.responseCls(self.connection.getresponse())
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 48, in __init__ raise Exception(self.parse_error())
> Exception: InvalidInstanceID.NotFound: The instance ID 'i-55c02c12' does not exist
> Note that the instance was in fact started.
> I also captured the libcloud request/response log and can provide that if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Resolved] (LIBCLOUD-104) EC2 driver automatic tag creation
can fail
Posted by "Tomaz Muraus (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tomaz Muraus resolved LIBCLOUD-104.
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.6.1
> EC2 driver automatic tag creation can fail
> ------------------------------------------
>
> Key: LIBCLOUD-104
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-104
> Project: Libcloud
> Issue Type: Bug
> Components: Compute
> Affects Versions: 0.5.2
> Reporter: David LaBissoniere
> Fix For: 0.6.1
>
>
> The EC2 driver automatically creates tags for new nodes in create_node(). Rarely, this can fail with an InvalidInstanceID.NotFound error from EC2. This error bubbles up to the create_node caller and prevents the new node information from being returned.
> I've seen similar issues before on EC2 when doing a instance query right after a launch. The new node isn't always included in the result, perhaps due to some caching or data propagation issue. I suspect the same thing is happening in the tag creation API. Perhaps the ex_create_tags calls could be wrapped in a try/except with this error logged and ignored.
> Here is a stack trace:
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 862, in create_node self.ex_create_tags(node=node, tags=tags)
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 672, in ex_create_tags
> params=params.copy()).object File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 397, in request
> response = self.responseCls(self.connection.getresponse())
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 48, in __init__ raise Exception(self.parse_error())
> Exception: InvalidInstanceID.NotFound: The instance ID 'i-55c02c12' does not exist
> Note that the instance was in fact started.
> I also captured the libcloud request/response log and can provide that if needed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-104) EC2 driver automatic tag creation
can fail
Posted by "Tim Freeman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090404#comment-13090404 ]
Tim Freeman commented on LIBCLOUD-104:
--------------------------------------
This is not an uncommon problem, I just ran into it, only noticing the leak because I was running an experiment. Thanks.
> EC2 driver automatic tag creation can fail
> ------------------------------------------
>
> Key: LIBCLOUD-104
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-104
> Project: Libcloud
> Issue Type: Bug
> Components: Compute
> Affects Versions: 0.5.2
> Reporter: David LaBissoniere
> Priority: Minor
>
> The EC2 driver automatically creates tags for new nodes in create_node(). Rarely, this can fail with an InvalidInstanceID.NotFound error from EC2. This error bubbles up to the create_node caller and prevents the new node information from being returned.
> I've seen similar issues before on EC2 when doing a instance query right after a launch. The new node isn't always included in the result, perhaps due to some caching or data propagation issue. I suspect the same thing is happening in the tag creation API. Perhaps the ex_create_tags calls could be wrapped in a try/except with this error logged and ignored.
> Here is a stack trace:
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 862, in create_node self.ex_create_tags(node=node, tags=tags)
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 672, in ex_create_tags
> params=params.copy()).object File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 397, in request
> response = self.responseCls(self.connection.getresponse())
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 48, in __init__ raise Exception(self.parse_error())
> Exception: InvalidInstanceID.NotFound: The instance ID 'i-55c02c12' does not exist
> Note that the instance was in fact started.
> I also captured the libcloud request/response log and can provide that if needed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Updated] (LIBCLOUD-104) EC2 driver automatic tag creation
can fail
Posted by "Tomaz Muraus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tomaz Muraus updated LIBCLOUD-104:
----------------------------------
Priority: Major (was: Minor)
> EC2 driver automatic tag creation can fail
> ------------------------------------------
>
> Key: LIBCLOUD-104
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-104
> Project: Libcloud
> Issue Type: Bug
> Components: Compute
> Affects Versions: 0.5.2
> Reporter: David LaBissoniere
>
> The EC2 driver automatically creates tags for new nodes in create_node(). Rarely, this can fail with an InvalidInstanceID.NotFound error from EC2. This error bubbles up to the create_node caller and prevents the new node information from being returned.
> I've seen similar issues before on EC2 when doing a instance query right after a launch. The new node isn't always included in the result, perhaps due to some caching or data propagation issue. I suspect the same thing is happening in the tag creation API. Perhaps the ex_create_tags calls could be wrapped in a try/except with this error logged and ignored.
> Here is a stack trace:
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 862, in create_node self.ex_create_tags(node=node, tags=tags)
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 672, in ex_create_tags
> params=params.copy()).object File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 397, in request
> response = self.responseCls(self.connection.getresponse())
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 48, in __init__ raise Exception(self.parse_error())
> Exception: InvalidInstanceID.NotFound: The instance ID 'i-55c02c12' does not exist
> Note that the instance was in fact started.
> I also captured the libcloud request/response log and can provide that if needed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-104) EC2 driver automatic tag creation
can fail
Posted by "Tomaz Muraus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070111#comment-13070111 ]
Tomaz Muraus commented on LIBCLOUD-104:
---------------------------------------
Yeah, that it is a know problem, thanks for reporting it though.
We need a better of way of handling all the operations which involve multiple steps, possibly also adding support for retrying some action.
I think we already had open ticket with this problem with more details, but I can't find it atm.
> EC2 driver automatic tag creation can fail
> ------------------------------------------
>
> Key: LIBCLOUD-104
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-104
> Project: Libcloud
> Issue Type: Bug
> Components: Compute
> Affects Versions: 0.5.2
> Reporter: David LaBissoniere
> Priority: Minor
>
> The EC2 driver automatically creates tags for new nodes in create_node(). Rarely, this can fail with an InvalidInstanceID.NotFound error from EC2. This error bubbles up to the create_node caller and prevents the new node information from being returned.
> I've seen similar issues before on EC2 when doing a instance query right after a launch. The new node isn't always included in the result, perhaps due to some caching or data propagation issue. I suspect the same thing is happening in the tag creation API. Perhaps the ex_create_tags calls could be wrapped in a try/except with this error logged and ignored.
> Here is a stack trace:
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 862, in create_node self.ex_create_tags(node=node, tags=tags)
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> compute/drivers/ec2.py", line 672, in ex_create_tags
> params=params.copy()).object File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 397, in request
> response = self.responseCls(self.connection.getresponse())
> File "/home/cc/app-venv/lib/python2.6/site-packages/apache_libcloud-0.5.2-py2.6.egg/libcloud/
> common/base.py", line 48, in __init__ raise Exception(self.parse_error())
> Exception: InvalidInstanceID.NotFound: The instance ID 'i-55c02c12' does not exist
> Note that the instance was in fact started.
> I also captured the libcloud request/response log and can provide that if needed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira