[jira] [Commented] (ARROW-7857) [Python] Failing test with pandas master for extension type conversion

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-7857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17036839#comment-17036839 ] 

Joris Van den Bossche commented on ARROW-7857:
----------------------------------------------

The test can easily be fixed on our side, but it would also be solved by implementing casting for extension types: ARROW-7858

> [Python] Failing test with pandas master for extension type conversion
> ----------------------------------------------------------------------
>
>                 Key: ARROW-7857
>                 URL: https://issues.apache.org/jira/browse/ARROW-7857
>             Project: Apache Arrow
>          Issue Type: Test
>          Components: Python
>            Reporter: Joris Van den Bossche
>            Assignee: Joris Van den Bossche
>            Priority: Major
>
> The pandas master test build has one failure
> {code}
> _______________ test_conversion_extensiontype_to_extensionarray ________________
> monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fcd6c580bd0>
>     def test_conversion_extensiontype_to_extensionarray(monkeypatch):
>         # converting extension type to linked pandas ExtensionDtype/Array
>         import pandas.core.internals as _int
>     
>         storage = pa.array([1, 2, 3, 4], pa.int64())
>         arr = pa.ExtensionArray.from_storage(MyCustomIntegerType(), storage)
>         table = pa.table({'a': arr})
>     
>         if LooseVersion(pd.__version__) < "0.26.0.dev":
>             # ensure pandas Int64Dtype has the protocol method (for older pandas)
>             monkeypatch.setattr(
>                 pd.Int64Dtype, '__from_arrow__', _Int64Dtype__from_arrow__,
>                 raising=False)
>     
>         # extension type points to Int64Dtype, which knows how to create a
>         # pandas ExtensionArray
> >       result = table.to_pandas()
> opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py:3560: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> pyarrow/ipc.pxi:559: in pyarrow.lib.read_message
>     ???
> pyarrow/table.pxi:1369: in pyarrow.lib.Table._to_pandas
>     ???
> opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:764: in table_to_blockmanager
>     blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes)
> opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:1102: in _table_to_blocks
>     for item in result]
> opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:1102: in <listcomp>
>     for item in result]
> opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:723: in _reconstruct_block
>     pd_ext_arr = pandas_dtype.__from_arrow__(arr)
> opt/conda/envs/arrow/lib/python3.7/site-packages/pandas/core/arrays/integer.py:108: in __from_arrow__
>     array = array.cast(pyarrow_type)
> pyarrow/table.pxi:240: in pyarrow.lib.ChunkedArray.cast
>     ???
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> >   ???
> E   pyarrow.lib.ArrowNotImplementedError: No cast implemented from extension<arrow.py_extension_type> to int64
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)