You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Carl Boettiger (Jira)" <ji...@apache.org> on 2021/12/14 16:54:00 UTC

[jira] [Created] (ARROW-15104) write_parquet() / write_csv_arrow() cannot stream a dataset object back to S3

Carl Boettiger created ARROW-15104:
--------------------------------------

             Summary: write_parquet() / write_csv_arrow() cannot stream a dataset object back to S3
                 Key: ARROW-15104
                 URL: https://issues.apache.org/jira/browse/ARROW-15104
             Project: Apache Arrow
          Issue Type: Bug
          Components: R
    Affects Versions: 6.0.1
            Reporter: Carl Boettiger


It seems we cannot send a dataset (table) object opened with `open_dataset()` into `write_parquet()` or `write_csv_arrow()`.  I realize that the former is a 'multi-file' representation and the latter two are not, but as both represent a single table (single schema) it seems like this should still work.  (It also looks like write_csv_arrow() gives us more control over formatting details than write_dataset() )



--
This message was sent by Atlassian Jira
(v8.20.1#820001)