You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@libcloud.apache.org by "John Carr (JIRA)" <ji...@apache.org> on 2012/11/12 17:37:13 UTC
[dev] [jira] [Created] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
John Carr created LIBCLOUD-257:
----------------------------------
Summary: S3 get_object hangs when object doesn't exist but only on python 2.6
Key: LIBCLOUD-257
URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
Project: Libcloud
Issue Type: Bug
Components: Storage
Environment: Python2.6
Reporter: John Carr
A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
Posted by "Tomaz Muraus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495448#comment-13495448 ]
Tomaz Muraus commented on LIBCLOUD-257:
---------------------------------------
Interesting, thanks for the patch.
I'll try to dig into and replicate the problem with and without the patch.
> S3 get_object hangs when object doesn't exist but only on python 2.6
> --------------------------------------------------------------------
>
> Key: LIBCLOUD-257
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Python2.6
> Reporter: John Carr
> Attachments: fix_s3_hanging_python26.diff
>
>
> A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
Posted by "John Carr (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495790#comment-13495790 ]
John Carr commented on LIBCLOUD-257:
------------------------------------
Sorry about the formatting, it was a quick refactor of the monkey patch I ended up using to get it working for me to show the nature of the problem. While I tested the original monkey patch quite a bit, in my haste I didn't test the monkey-patch-less version. I can attach a new patch if you like, but you have probably already fixed it locally if you were able to recreate the problem and prove this fixes it?
While I agree that making this patch only active on 2.6 is probably a good idea, I just wanted to stress that this isn't some weird hack to make python 2.6 work. Calling self.close() if the method is HEAD is the right thing to do. Or at least, thats what python 2.7 and later do - this patch it is essentially a backport of the behaviour found on all later pythons.
What is the preferred way to hide something like this from python 2.7 and later in libcloud?
> S3 get_object hangs when object doesn't exist but only on python 2.6
> --------------------------------------------------------------------
>
> Key: LIBCLOUD-257
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Python2.6
> Reporter: John Carr
> Attachments: fix_s3_hanging_python26.diff
>
>
> A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
Posted by "Tomaz Muraus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495888#comment-13495888 ]
Tomaz Muraus commented on LIBCLOUD-257:
---------------------------------------
I've made the changes I mentioned above and merged the patch into trunk - http://svn.apache.org/viewvc?rev=1408567&view=rev.
Thanks again for tracking down this issue and submitting patch.
> S3 get_object hangs when object doesn't exist but only on python 2.6
> --------------------------------------------------------------------
>
> Key: LIBCLOUD-257
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Python2.6
> Reporter: John Carr
> Attachments: fix_s3_hanging_python26.diff
>
>
> A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
Posted by "Tomaz Muraus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495598#comment-13495598 ]
Tomaz Muraus commented on LIBCLOUD-257:
---------------------------------------
I've checked this patch and noticed some issues:
1. Indentation in the HTTPResponse class is messed up
2. Calling self.close if method is HEAD every time doesn't seem like a right this to do. If this issue is specific to Python 2.6, we should only do it in this specific case.
> S3 get_object hangs when object doesn't exist but only on python 2.6
> --------------------------------------------------------------------
>
> Key: LIBCLOUD-257
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Python2.6
> Reporter: John Carr
> Attachments: fix_s3_hanging_python26.diff
>
>
> A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Resolved] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
Posted by "Tomaz Muraus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tomaz Muraus resolved LIBCLOUD-257.
-----------------------------------
Resolution: Fixed
> S3 get_object hangs when object doesn't exist but only on python 2.6
> --------------------------------------------------------------------
>
> Key: LIBCLOUD-257
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Python2.6
> Reporter: John Carr
> Attachments: fix_s3_hanging_python26.diff
>
>
> A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Updated] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
Posted by "John Carr (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Carr updated LIBCLOUD-257:
-------------------------------
Attachment: fix_s3_hanging_python26.diff
This is how I fixed it.
> S3 get_object hangs when object doesn't exist but only on python 2.6
> --------------------------------------------------------------------
>
> Key: LIBCLOUD-257
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Python2.6
> Reporter: John Carr
> Attachments: fix_s3_hanging_python26.diff
>
>
> A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[dev] [jira] [Commented] (LIBCLOUD-257) S3 get_object hangs when object
doesn't exist but only on python 2.6
Posted by "Tomaz Muraus (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LIBCLOUD-257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495802#comment-13495802 ]
Tomaz Muraus commented on LIBCLOUD-257:
---------------------------------------
Ah, thanks for the explanation. I'll have a look at Python source code and see how HEAD requests are handled in 2.6 and 2.7.
As far as only running the code in some versions, we use sys.version_info.
> S3 get_object hangs when object doesn't exist but only on python 2.6
> --------------------------------------------------------------------
>
> Key: LIBCLOUD-257
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-257
> Project: Libcloud
> Issue Type: Bug
> Components: Storage
> Environment: Python2.6
> Reporter: John Carr
> Attachments: fix_s3_hanging_python26.diff
>
>
> A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira