[dev] Storage: data type for object content?

[Torf <ma...@torf.cc>]

Hi everybody,

as discussed on the users mailing list I'm working on a problem
regarding LocalStorageDriver and binary object data (see
https://issues.apache.org/jira/browse/LIBCLOUD-725). To fix this, I need
to know if there is a fixed data type that storage drivers use for
object data. I couldn't find anything in the documentation regarding
this question and since this is my first time working on LibCloud
internals I want to double-check that I understand things correctly.

I'm assuming that a fixed data type for object data is desired --
otherwise, we would either need a way to tell
driver.download_object_as_stream the expected data type or perform data
type checks on its return value.

Candidate data types for object data seem to be bytes (Python 2: str,
Python 3: bytes) and text (Python 2: unicode, Python 3: str). However,
since object content can be both text-based or in a binary format using
bytes is the only reasonable choice.

This means that driver.upload_object_via_stream expects an iterator that
yields chunks of bytes, and that driver.download_object_as_stream also
yields chunks of bytes. Similarly, driver.upload_object and
driver.download_object must open the source/destination files in binary
mode.

Please tell me if this line of reasoning is consistent with LibCloud
standards.


Regards,
Torf