You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@libcloud.apache.org by GitBox <gi...@apache.org> on 2022/05/05 13:06:47 UTC
[GitHub] [libcloud] RunOrVeith opened a new issue, #1682: Google storage access is broken for public blobs
RunOrVeith opened a new issue, #1682:
URL: https://github.com/apache/libcloud/issues/1682
## Summary
When requesting a blob from google storage that is public (it works for private buckets), Google seems to have suddenly stopped sending the `etag` header. Unfortunately, this is required in `_headers_to_object`, which now crashes.
This means no files on a public bucket can be accessed any more at the moment.
## Detailed Information
```python
from libcloud.storage.providers import Provider, get_driver
key = "" # add your key here
secret = "" # add your secret here
# Make sure this points to a public blob, it works for private buckets
container_name = "" # add a container name here
object_name = "" # add the object name here
driver_type = get_driver(provider=Provider.GOOGLE_STORAGE)
driver = driver_type(key=key, secret=secret)
driver.get_object(
container_name=container_name,
object_name=object_name
)
```
Results in the following error:
```
Traceback (most recent call last):
File "/home/veith/.config/JetBrains/PyCharm2022.1/scratches/scratch_1.py", line 12, in <module>
driver.get_object(
File "/home/veith/Projects/.venv/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line 412, in get_object
obj = self._headers_to_object(
File "/home/veith/Projects/.venv/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line 1195, in _headers_to_object
hash = headers["etag"].replace('"', "")
KeyError: 'etag'
```
The problem are the first 2 lines in s3.py:
```python
def _headers_to_object(self, object_name, container, headers):
hash = headers["etag"].replace('"', "") # expects etag to be present and will crash
extra = {"content_type": headers["content-type"], "etag": headers["etag"]}
meta_data = {}
```
Libcloud 3.5.1, Ubuntu 18.04, Python 3.8
I will try to create a fix ASAP
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] RunOrVeith commented on issue #1682: Google storage sometimes does not send etag
Posted by GitBox <gi...@apache.org>.
RunOrVeith commented on issue #1682:
URL: https://github.com/apache/libcloud/issues/1682#issuecomment-1119352829
[I created an issue in the google issue tracker for this](https://issuetracker.google.com/issues/231495126)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] asfgit closed issue #1682: Google storage sometimes does not send etag
Posted by GitBox <gi...@apache.org>.
asfgit closed issue #1682: Google storage sometimes does not send etag
URL: https://github.com/apache/libcloud/issues/1682
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] RunOrVeith commented on issue #1682: Google storage sometimes does not send etag
Posted by GitBox <gi...@apache.org>.
RunOrVeith commented on issue #1682:
URL: https://github.com/apache/libcloud/issues/1682#issuecomment-1119352253
This affects any images uploaded with gzip encoding.
So to reproduce, upload the file that you check with the code above like this
`gsutil cp -Z ~/Pictures/some-image.png gs://my-bucket/some-image-gzip.png`.
It works if you upload without gzip encoding:
`gsutil cp ~/Pictures/some-image.png gs://my-bucket/some-image-no-gzip.png`
It works for non-image files regardless of gzip encoding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1682: Google storage sometimes does not send etag
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1682:
URL: https://github.com/apache/libcloud/issues/1682#issuecomment-1120010524
Thanks for reporting this.
I also did some quick testing myself and I was indeed only able to reproduce it when content encoding is gzip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] RunOrVeith commented on issue #1682: Google storage access is broken for public blobs
Posted by GitBox <gi...@apache.org>.
RunOrVeith commented on issue #1682:
URL: https://github.com/apache/libcloud/issues/1682#issuecomment-1118589649
[Also created an issue in the google issue tracker](https://issuetracker.google.com/issues/231495126)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org