You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Joris Van den Bossche (Jira)" <ji...@apache.org> on 2022/04/21 18:33:00 UTC
[jira] [Updated] (ARROW-16240) [Python] Support row_group_size/chunk_size keyword in pq.write_to_dataset with use_legacy_dataset=False
[ https://issues.apache.org/jira/browse/ARROW-16240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joris Van den Bossche updated ARROW-16240:
------------------------------------------
Summary: [Python] Support row_group_size/chunk_size keyword in pq.write_to_dataset with use_legacy_dataset=False (was: [Python] _create_dataset_for_fragments() helper function needs to be updated)
> [Python] Support row_group_size/chunk_size keyword in pq.write_to_dataset with use_legacy_dataset=False
> -------------------------------------------------------------------------------------------------------
>
> Key: ARROW-16240
> URL: https://issues.apache.org/jira/browse/ARROW-16240
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Alenka Frim
> Priority: Major
>
> {{_create_dataset_for_fragments() }}helper function in test_dataset.py needs to be updated to reflect the changes in the {{write_to_dataset}} in ARROW-16122 : The default for {{use_legacy_dataset}} keyword will be set to False but the {{dataset.write_dataset(..)}} doesn't support the parquet {{row_group_size}} keyword.
> See discussion: [https://github.com/apache/arrow/pull/12811#discussion_r845304218]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)