You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "meta-ks (via GitHub)" <gi...@apache.org> on 2023/10/19 10:31:46 UTC
[I] DataFrame to Arrow Table Conversion issue with Tuple datatype [arrow]
meta-ks opened a new issue, #38344:
URL: https://github.com/apache/arrow/issues/38344
### Describe the bug, including details regarding any error messages, version, and platform.
While attempting to convert a DataFrame containing a column with tuples to an Arrow Table and back to a DataFrame, we encountered a discrepancy (expected tuple, got np.ndarray) between the original DataFrame and the DataFrame obtained from the Arrow Table.
Steps to Reproduce:
1. Create a DataFrame with a column that contains tuples.
2. Convert the DataFrame to an Arrow Table using pa.Table.from_pandas().
3. Convert the Arrow Table back to a DataFrame using table.to_pandas().
4. Compare the original DataFrame with the DataFrame obtained from the Arrow Table.
Expected Results:
The DataFrames before and after conversion should be identical.
Actual Results:
The DataFrames differ.
```
import pandas as pd
import pyarrow as pa
data = {'tup': [(1, 2), (3, 4), (5, 6)]}
df = pd.DataFrame(data)
table = pa.Table.from_pandas(df)
df_back = table.to_pandas()
print("Original DataFrame:")
print(df)
print("\nDataFrame converted from Arrow Table:")
print(df_back)
print("\nAre the DataFrames equal?", df.equals(df_back))
```

### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org