[jira] [Created] (ARROW-15375) Parquet write_to_dataset leads to partial write when unsupported datatype is passed in table

["Chandrasekaran Anirudh Bhardwaj (Jira)" <ji...@apache.org>]

Chandrasekaran Anirudh Bhardwaj created ARROW-15375:
-------------------------------------------------------

             Summary: Parquet write_to_dataset leads to partial write when unsupported datatype is passed in table 
                 Key: ARROW-15375
                 URL: https://issues.apache.org/jira/browse/ARROW-15375
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
         Environment: Linux (Ubuntu 20.04)
            Reporter: Chandrasekaran Anirudh Bhardwaj


Trying to save unsupported datatype in parquet using pyarrow.write_to_dataset results in a partial folder and file write to disk.

 
{code:java}
import pandas as pd
import numpy as np
import pyarrow as pa
import pyarrow.parquet as pq

data = np.arange(2, 10, dtype=np.float16) 
df = pd.DataFrame(data=data, columns=['fp16'])
table=pa.Table.from_pandas(df)

pq.write_to_dataset(table=table, root_path='./fp16_fail_dataset'){code}
 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)