You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Joris Van den Bossche (Jira)" <ji...@apache.org> on 2020/10/30 12:00:00 UTC
[jira] [Updated] (ARROW-10419) [C++] Add max_rows parameter to csv
ReadOptions
[ https://issues.apache.org/jira/browse/ARROW-10419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joris Van den Bossche updated ARROW-10419:
------------------------------------------
Summary: [C++] Add max_rows parameter to csv ReadOptions (was: Add max_rows parameter to pyarrow.csv.ReadOptions)
> [C++] Add max_rows parameter to csv ReadOptions
> -----------------------------------------------
>
> Key: ARROW-10419
> URL: https://issues.apache.org/jira/browse/ARROW-10419
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Python
> Reporter: Marc Garcia
> Priority: Major
>
> I'm trying to read only the first 1,000 rows of a huge CSV with PyArrow.
> I don't see a way to do this with Arrow. I guess it should be easy to implement by adding a `max_rows` parameter to pyarrow.csv.ReadOptions.
> After reading the first 1,000, it should be possible to load the next 1,000 (or any other chunk) by using both the new `max_rows` together with `skip_rows` (e.g. `pyarrow.csv.read_csv(path, pyarrow.csv.ReadOption(skip_rows=1_000, max_rows=1_000)` would read from 1,000 to 2,000).
> Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)