You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/11/05 07:24:00 UTC
[jira] [Updated] (ARROW-13756) [Python] Error in pandas conversion
for datetimetz column index
[ https://issues.apache.org/jira/browse/ARROW-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated ARROW-13756:
-----------------------------------
Labels: pandas pull-request-available (was: pandas)
> [Python] Error in pandas conversion for datetimetz column index
> ---------------------------------------------------------------
>
> Key: ARROW-13756
> URL: https://issues.apache.org/jira/browse/ARROW-13756
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 5.0.0
> Environment: Ubuntu 21.04
> Reporter: Andreas Wolf
> Assignee: Alenka Frim
> Priority: Major
> Labels: pandas, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following code fails with:
> {code:java}
> File "[...]/lib/python3.8/site-packages/pyarrow/pandas_compat.py", line 1052, in _pandas_type_to_numpy_type
> return np.dtype(pandas_type)
> TypeError: data type 'datetimetz' not understood{code}
> Sample:
> {code:java}
> def run():
> filename = "test.parquet"
> df = pd.DataFrame(
> data=range(31),
> columns=list("A"),
> index=pd.date_range("2021-01-01", "2021-01-31", freq="D", tz="CET"),
> ).T
> table = pa.Table.from_pandas(df)
> pq.write_to_dataset(table, root_path=filename)
> result = pq.read_table(filename).to_pandas()
> return result
> if __name__ == "__main__":
> run()
> {code}
> The code tries to store a dataframe where the columns are timezone aware datetimes.
> _Observations_:
> If I remove the *.T* at the end of the dataframe, so that the datatime index are rows it is working (but not what I want).
> If I remove the timezone information *tz="CET"* the code is working.
> I assume this bug is related to [Error in pandas conversion for datetimetz row index|https://issues.apache.org/jira/browse/ARROW-1958]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)