[GitHub] [libcloud] yagodorea opened a new issue, #1802: "UnicodeEncodeError: 'latin-1' codec can't encode character, ordinal not in range" on GCS upload_object_via_stream

[GitBox <gi...@apache.org>]

yagodorea opened a new issue, #1802:
URL: https://github.com/apache/libcloud/issues/1802

   ## Summary
   
   I'm trying to upload some CSV files to GCS using libcloud, today I this error:
   
   ```log
   UnicodeEncodeError: 'latin-1' codec can't encode character '\u201c' in position 628: ordinal not in range(256)
   ```
   
   ## Detailed Information
   
   The encode character and position change with each run, even though the files don't change. I'm not sure what's causing that, CSV files are basically plaintext.
   
   Here's the full stack (some internal code ommitted):
   
   ```log
   Traceback (most recent call last):
     [...]
       self.__container.upload_object_via_stream(stream, name)  # type: ignore
     File "/usr/local/lib/python3.8/site-packages/libcloud/storage/base.py", line 227, in upload_object_via_stream
       return self.driver.upload_object_via_stream(
     File "/usr/local/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line 843, in upload_object_via_stream
       return self._put_object(
     File "/usr/local/lib/python3.8/site-packages/libcloud/storage/drivers/s3.py", line 1007, in _put_object
       result_dict = self._upload_object(
     File "/usr/local/lib/python3.8/site-packages/libcloud/storage/base.py", line 874, in _upload_object
       response = self.connection.request(
     File "/usr/local/lib/python3.8/site-packages/libcloud/common/base.py", line 659, in request
       return request_to_be_executed(
     File "/usr/local/lib/python3.8/site-packages/libcloud/common/base.py", line 678, in _retryable_request
       self.connection.prepared_request(
     File "/usr/local/lib/python3.8/site-packages/libcloud/http.py", line 260, in prepared_request
       self.response = self.session.send(
     File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 645, in send
       r = adapter.send(request, **kwargs)
     File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 440, in send
       resp = conn.urlopen(
     File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
       httplib_response = self._make_request(
     File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 398, in _make_request
       conn.request(method, url, **httplib_request_kw)
     File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 239, in request
       super(HTTPConnection, self).request(method, url, body=body, headers=headers)
     File "/usr/local/lib/python3.8/http/client.py", line 1256, in request
       self._send_request(method, url, body, headers, encode_chunked)
     File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request
       self.endheaders(body, encode_chunked=encode_chunked)
     File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders
       self._send_output(message_body, encode_chunked=encode_chunked)
     File "/usr/local/lib/python3.8/http/client.py", line 1040, in _send_output
       for chunk in chunks:
     File "/usr/local/lib/python3.8/http/client.py", line 999, in _read_readable
       datablock = datablock.encode("iso-8859-1")
   UnicodeEncodeError: 'latin-1' codec can't encode character '\u2019' in position 5251: ordinal not in range(256)
   ```
   
   I'm running this from inside a k8s pod in a GKE cluster, trying to write to a GCS bucket.
   
   ```
   apache-libcloud version: 3.5.1
   python version: 3.8.15
   pod image: python:3.8-slim
   ```
   
   I'm not sure what are the steps to reproduce it. It's a CSV file generated by pulling some BigQuery schema metadata. It works locally, but when I ran it in the pod it raised this error.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org