[jira] [Updated] (ARROW-16436) [C++] Datasets ignores CSV autogenerate_column_names during discovery

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-16436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-16436:
-----------------------------------
    Labels: good-first-issue pull-request-available  (was: good-first-issue)

> [C++] Datasets ignores CSV autogenerate_column_names during discovery
> ---------------------------------------------------------------------
>
>                 Key: ARROW-16436
>                 URL: https://issues.apache.org/jira/browse/ARROW-16436
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++
>    Affects Versions: 7.0.0
>            Reporter: David Li
>            Assignee: Raúl Cumplido
>            Priority: Major
>              Labels: good-first-issue, pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Reproduction
> {code:python}
> import tempfile
> from pathlib import Path
> import pyarrow as pa
> import pyarrow.csv as csv
> import pyarrow.dataset as ds
> print("PyArrow version:", pa.__version__)
> ro = csv.ReadOptions(autogenerate_column_names=True)
> po = csv.ParseOptions()
> co = csv.ConvertOptions()
> file_format = ds.CsvFileFormat(read_options=ro, parse_options=po, convert_options=co)
> with tempfile.TemporaryDirectory() as td:
>     td = Path(td).resolve()
>     with (td / "test.csv").open("w") as sink:
>         sink.write("1,a,true,1\n")
>     dataset = ds.dataset(str(td), format=file_format)
>     print(dataset.to_table())
> {code}
> Result:
> {noformat}
> PyArrow version: 7.0.0
> Traceback (most recent call last):
>   File "/home/lidavidm/csvdemo.py", line 20, in <module>
>     dataset = ds.dataset(str(td), format=file_format)
>   File "/home/lidavidm/miniconda3/envs/arrow/lib/python3.10/site-packages/pyarrow/dataset.py", line 667, in dataset
>     return _filesystem_dataset(source, **kwargs)
>   File "/home/lidavidm/miniconda3/envs/arrow/lib/python3.10/site-packages/pyarrow/dataset.py", line 422, in _filesystem_dataset
>     return factory.finish(schema)
>   File "pyarrow/_dataset.pyx", line 1680, in pyarrow._dataset.DatasetFactory.finish
>   File "pyarrow/error.pxi", line 143, in pyarrow.lib.pyarrow_internal_check_status
>   File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
> pyarrow.lib.ArrowInvalid: Error creating dataset. Could not read schema from '/tmp/tmp5rz0ipmm/test.csv': Could not open CSV input source '/tmp/tmp5rz0ipmm/test.csv': Invalid: CSV file contained multiple columns named 1. Is this a 'csv' file?
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)