[jira] [Commented] (ARROW-17766) Add option to only load specific columns from csv

["Marc Garcia (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17606331#comment-17606331 ] 

Marc Garcia commented on ARROW-17766:
-------------------------------------

Ok, my bad, seems like it was there, `include_columns` at the end of the `ConverterOptions` parameters, I couldn't find it after looking at all the options 3 times (just found it checking the source code): https://arrow.apache.org/docs/python/generated/pyarrow.csv.ConvertOptions.html#pyarrow.csv.ConvertOptions

> Add option to only load specific columns from csv
> -------------------------------------------------
>
>                 Key: ARROW-17766
>                 URL: https://issues.apache.org/jira/browse/ARROW-17766
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: Python
>            Reporter: Marc Garcia
>            Priority: Major
>
> I may be missing something, but after checking in detail the documentation of pyarrow, I can't find an option equivalent to `pandas.read_csv(..., usecols=['col_to_load_1', 'col_to_load_2'])`.
> This would be useful for example when loading CSV files with lots of columns where only few are needed. Having to load all the information in the CSV seems like a significant waste of resources.
> I guess a parameter `load_columns` could be added to `pyarrow.csv.ReadOptions`.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)