You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Uwe Korn (Jira)" <ji...@apache.org> on 2020/12/09 09:41:00 UTC
[jira] [Resolved] (ARROW-10855) [Python][Numpy] ArrowTypeError
after upgrading NumPy to 1.20.0rc1
[ https://issues.apache.org/jira/browse/ARROW-10855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Korn resolved ARROW-10855.
------------------------------
Fix Version/s: 3.0.0
Assignee: Uwe Korn
Resolution: Duplicate
Closing as a duplicate of https://issues.apache.org/jira/browse/ARROW-10833 which I will merge now.
> [Python][Numpy] ArrowTypeError after upgrading NumPy to 1.20.0rc1
> -----------------------------------------------------------------
>
> Key: ARROW-10855
> URL: https://issues.apache.org/jira/browse/ARROW-10855
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 2.0.0
> Environment: macOS Big Sur 11.0.1
> Reporter: Zhenghui Jin
> Assignee: Uwe Korn
> Priority: Major
> Fix For: 3.0.0
>
>
> After upgrading numpy to 1.20.0rc1 version, pandas .to_parquet() will raise ArrowTypeError.
> NumPy 1.19.4, Python 3.7.9, macos:
>
> {code:java}
> Python 3.7.9 (default, Nov 20 2020, 23:58:42)
> [Clang 12.0.0 (clang-1200.0.32.27)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy as np
> >>> import pandas as pd
> >>> np.__version__
> '1.19.4'
> >>> pd.DataFrame({'i': [1, 2, 3, np.nan]}, dtype='Int64').to_parquet('nullint.parquet')
> >>>
> {code}
> NumPy 1.20.0rc1, Python 3.7.9, macos:
> {code:java}
> Python 3.7.9 (default, Nov 20 2020, 23:58:42)
> [Clang 12.0.0 (clang-1200.0.32.27)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy as np
> >>> import pandas as pd
> >>> np.__version__
> '1.20.0rc1'
> >>> pd.DataFrame({'i': [1, 2, 3, np.nan]}, dtype='Int64').to_parquet('nullint.parquet')
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/local/lib/python3.7/site-packages/pandas/util/_decorators.py", line 199, in wrapper
> return func(*args, **kwargs)
> File "/usr/local/lib/python3.7/site-packages/pandas/core/frame.py", line 2372, in to_parquet
> **kwargs,
> File "/usr/local/lib/python3.7/site-packages/pandas/io/parquet.py", line 276, in to_parquet
> **kwargs,
> File "/usr/local/lib/python3.7/site-packages/pandas/io/parquet.py", line 101, in write
> table = self.api.Table.from_pandas(df, **from_pandas_kwargs)
> File "pyarrow/table.pxi", line 1394, in pyarrow.lib.Table.from_pandas
> File "/usr/local/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 588, in dataframe_to_arrays
> for c, f in zip(columns_to_convert, convert_fields)]
> File "/usr/local/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 588, in <listcomp>
> for c, f in zip(columns_to_convert, convert_fields)]
> File "/usr/local/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 574, in convert_column
> raise e
> File "/usr/local/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 568, in convert_column
> result = pa.array(col, type=type_, from_pandas=True, safe=safe)
> File "pyarrow/array.pxi", line 242, in pyarrow.lib.array
> File "pyarrow/array.pxi", line 110, in pyarrow.lib._handle_arrow_array_protocol
> File "/usr/local/lib/python3.7/site-packages/pandas/core/arrays/masked.py", line 227, in __arrow_array__
> return pa.array(self._data, mask=self._mask, type=type)
> File "pyarrow/array.pxi", line 292, in pyarrow.lib.array
> File "pyarrow/array.pxi", line 79, in pyarrow.lib._ndarray_to_array
> File "pyarrow/array.pxi", line 67, in pyarrow.lib._ndarray_to_type
> File "pyarrow/error.pxi", line 107, in pyarrow.lib.check_status
> pyarrow.lib.ArrowTypeError: ('Did not pass numpy.dtype object', 'Conversion failed for column i with type Int64')
> >>>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)