[jira] [Resolved] (ARROW-13756) [Python] Error in pandas conversion for datetimetz column index

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joris Van den Bossche resolved ARROW-13756.
-------------------------------------------
    Fix Version/s: 7.0.0
       Resolution: Fixed

Issue resolved by pull request 11619
[https://github.com/apache/arrow/pull/11619]

> [Python] Error in pandas conversion for datetimetz column index
> ---------------------------------------------------------------
>
>                 Key: ARROW-13756
>                 URL: https://issues.apache.org/jira/browse/ARROW-13756
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 5.0.0
>         Environment: Ubuntu 21.04
>            Reporter: Andreas Wolf
>            Assignee: Alenka Frim
>            Priority: Major
>              Labels: pandas, pull-request-available
>             Fix For: 7.0.0
>
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> The following code fails with:
> {code:java}
> File "[...]/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 1052, in _pandas_type_to_numpy_type
>  return np.dtype(pandas_type)
> TypeError: data type 'datetimetz' not understood{code}
> Sample:
> {code:java}
> def run():
>     filename = "test.parquet"
>     df = pd.DataFrame(
>         data=range(31),
>         columns=list("A"),
>         index=pd.date_range("2021-01-01", "2021-01-31", freq="D", tz="CET"),
>     ).T
>     table = pa.Table.from_pandas(df)
>     pq.write_to_dataset(table, root_path=filename)
>     result = pq.read_table(filename).to_pandas()
>     return result
> if __name__ == "__main__":
>     run()
> {code}
> The code tries to store a dataframe where the columns are timezone aware datetimes.
> _Observations_:
> If I remove the *.T* at the end of the dataframe, so that the datatime index are rows it is working (but not what I want).
> If I remove the timezone information *tz="CET"* the code is working.
> I assume this bug is related to [Error in pandas conversion for datetimetz row index|https://issues.apache.org/jira/browse/ARROW-1958]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)