You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "David Li (Jira)" <ji...@apache.org> on 2022/10/17 16:10:00 UTC

[jira] [Resolved] (ARROW-17991) [Python] pyarrow.dataset IPC format does not support compression

     [ https://issues.apache.org/jira/browse/ARROW-17991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Li resolved ARROW-17991.
------------------------------
    Resolution: Fixed

Issue resolved by pull request 14414
[https://github.com/apache/arrow/pull/14414]

> [Python] pyarrow.dataset IPC format does not support compression
> ----------------------------------------------------------------
>
>                 Key: ARROW-17991
>                 URL: https://issues.apache.org/jira/browse/ARROW-17991
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Joost Hoozemans
>            Assignee: Joost Hoozemans
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 11.0.0
>
>          Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> When trying to write an IPC dataset using pyarrow.dataset, it is not possible to pass a compression argument:
> Trying to pass a pyarrow.ipc.IpcWriteOptions object:
> >>> ds.write_dataset(f, "./thing.arrow", format=ds.IpcFileFormat(), file_options=ipc.IpcWriteOptions(compression='lz4'))
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/joost/.cache/pypoetry/virtualenvs/datalogistik-rL_l_suP-py3.8/lib/python3.8/site-packages/pyarrow/dataset.py", line 940, in write_dataset
>     if format != file_options.format:
> AttributeError: 'pyarrow.lib.IpcWriteOptions' object has no attribute 'format'
>  
> Alternatively, pyarrow.dataset.IpcFileFormat().make_write_options() does not support a compression parameter



--
This message was sent by Atlassian Jira
(v8.20.10#820010)