You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2018/10/29 20:38:00 UTC
[jira] [Commented] (ARROW-3646) [Python] Add convenience factories
to create IO streams
[ https://issues.apache.org/jira/browse/ARROW-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667727#comment-16667727 ]
Wes McKinney commented on ARROW-3646:
-------------------------------------
Could call them {{pyarrow.input_stream}} or {{pyarrow.output_stream}} or something like that, too. We should probably rename {{pa.open_file}} and {{pa.open_stream}} to something that is clear that they are IPC message files/streams
> [Python] Add convenience factories to create IO streams
> -------------------------------------------------------
>
> Key: ARROW-3646
> URL: https://issues.apache.org/jira/browse/ARROW-3646
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Affects Versions: 0.11.0
> Reporter: Antoine Pitrou
> Priority: Major
> Fix For: 0.12.0
>
>
> Currently, creating IO streams requires invoking various constructors with irregular names. It would be nice to expose a higher-level interface.
> For example:
> {code:python}
> def open_reader(source, compression='detect'):
> """
> Create an Arrow input stream.
> Parameters
> ----------
> source: str, Path, buffer, file-like object, ...
> The source to open for reading
> compression: str or None
> The compression algorithm to use for on-the-fly decompression.
> If 'detect' and source is a file path, then compression will be
> chosen based on the file extension.
> If None, no compression will be applied.
> Otherwise, a well-known algorithm name must be supplied (e.g. "gzip")
> """
> def open_writer(source, compression='detect'):
> """
> Create an Arrow output stream.
> Parameters
> ----------
> source: str, Path, buffer, file-like object, ...
> The source to open for writing
> compression: str or None
> The compression algorithm to use for on-the-fly compression.
> If 'detect' and source is a file path, then compression will be
> chosen based on the file extension.
> If None, no compression will be applied.
> Otherwise, a well-known algorithm name must be supplied (e.g. "gzip")
> """
> {code}
> Thoughts?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)