[jira] [Commented] (ARROW-17535) [Python] List arrays aren't supported in to_pandas calls

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612532#comment-17612532 ] 

Joris Van den Bossche commented on ARROW-17535:
-----------------------------------------------

There is currently an open PR (https://github.com/apache/arrow/pull/14238) that addresses this (partly) by just using the storage array conversion. At the moment this doesn't take into account that the ExtensionType might define a custom conversion to numpy and/or pandas in Python. But my question: are we OK with for now falling back to storage array conversion? 

On the one hand, that would be consistent with StructArray, where we also fall back to the storage array at the moment. On the other hand, if we want to solve this more "properly" later, that would mean another change in behaviour. 

> [Python] List<Extension> arrays aren't supported in to_pandas calls
> -------------------------------------------------------------------
>
>                 Key: ARROW-17535
>                 URL: https://issues.apache.org/jira/browse/ARROW-17535
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, Python
>            Reporter: Micah Kornfield
>            Priority: Minor
>
> EXTENSION is not in the list of types allowed.  I think in order to enable EXTENSION we need to be able to call to_pylist or similar on the original extension array from C++ code, in case there were user provided overrides.  Off the top of my head one way of doing this would be to pass through an additional std::unorderd_map<Array*, PyObject*> where PyObject is the bound to_pylist python function.  Are there other alternative that might be cleaner?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)