[jira] [Commented] (ARROW-17839) Cannot create RecordBatch with nested struct containing extension type

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17609968#comment-17609968 ] 

Joris Van den Bossche commented on ARROW-17839:
-----------------------------------------------

[~mavam] Thanks for the report and the nice example!

So what is missing here is the automatic cast from the storage type to the extension type. This is currently being tackled in https://github.com/apache/arrow/pull/14106 (ARROW-14500). 

Testing your example with the branch of https://github.com/apache/arrow/pull/14106, it works for me (with one caveat that you need to register the extension types (eg {{pa.register_extension_type(subnet_type)}}) to preserve them in an IPC roundtrip.

> Cannot create RecordBatch with nested struct containing extension type
> ----------------------------------------------------------------------
>
>                 Key: ARROW-17839
>                 URL: https://issues.apache.org/jira/browse/ARROW-17839
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 9.0.0
>         Environment: macOS 12.5.1 on an Apple M1 Ultra.
>            Reporter: Matthias Vallentin
>            Priority: Blocker
>         Attachments: example.py
>
>
> I'm running into the following issue:
> {code:java}
> pyarrow.lib.ArrowNotImplementedError: Unsupported cast to extension<vast.address<AddressType>> from fixed_size_binary[16]{code}
> Use case: I want to create a record batch that contains this type:
> {code:java}
> pa.struct([("address", AddressType()), ("length", pa.uint8())]){code}
> Here, {{AddressType}} is an extension type that models an IP address ({{{}pa.binary(16){}}}).
> Please find attached a self-contained example that illustrates the issue.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)