[jira] [Created] (ARROW-17046) [Python] pyarrow.parquet.write_to_dataset fails to pass kwargs to write_table function

["Amir Khosroshahi (Jira)" <ji...@apache.org>]

Amir Khosroshahi created ARROW-17046:
----------------------------------------

             Summary: [Python] pyarrow.parquet.write_to_dataset fails to pass kwargs to write_table function
                 Key: ARROW-17046
                 URL: https://issues.apache.org/jira/browse/ARROW-17046
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
    Affects Versions: 8.0.0
            Reporter: Amir Khosroshahi


According to PyArrow 8.0.0 [documentation|https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_to_dataset.html] {{kwargs}} is "Additional kwargs for {{write_table}} function." However when I try to pass for example flavor as an additional argument to the underlying write_table I get the following error

 
{code:java}
TypeError: unexpected parquet write option: flavor{code}
 

This used to work in PyArrow as late as 7.0.0 but started to break in 8.0.0.

Minimal example to reproduce the error

 
{code:java}
import pyarrow as pa
import pandas as pd
import pyarrow.parquet as pq

df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})
tb = pa.Table.from_pandas(df)
pq.write_to_dataset(tb, "test.parquet", flavor="spark") {code}
 

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)