You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/03/26 20:03:00 UTC

[jira] [Updated] (ARROW-15643) [C++] Kernel to select subset of fields of a StructArray

     [ https://issues.apache.org/jira/browse/ARROW-15643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-15643:
-----------------------------------
    Labels: kernel pull-request-available  (was: kernel)

> [C++] Kernel to select subset of fields of a StructArray
> --------------------------------------------------------
>
>                 Key: ARROW-15643
>                 URL: https://issues.apache.org/jira/browse/ARROW-15643
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: C++
>            Reporter: Joris Van den Bossche
>            Assignee: Dhruv Vats
>            Priority: Major
>              Labels: kernel, pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Triggered by https://stackoverflow.com/questions/71035754/pyarrow-drop-a-column-in-a-nested-structure. I thought there was already an issue about this, but don't directly find one.
> Assume you have a struct array with some fields:
> {code}
> >>> arr = pa.StructArray.from_arrays([[1, 2, 3]]*3, names=['a', 'b', 'c'])
> >>> arr.type
> StructType(struct<a: int64, b: int64, c: int64>)
> {code}
> We have a kernel to select a single child field:
> {code}
> >>> pc.struct_field(arr, [0])
> <pyarrow.lib.Int64Array object at 0x7ffa9e229940>
> [
>   1,
>   2,
>   3
> ]
> {code}
> But if you want to subset the StructArray to some of its fields, resulting in a new StructArray, that's not possible with {{struct_field}}, and doing this manually is a bit cumbersome:
> {code}
> >>> fields = ['a', 'c']
> >>> arrays = [arr.field(n) for n in fields]
> >>> arr_subset = pa.StructArray.from_arrays(arrays, names=fields)
> >>> arr_subset.type
> StructType(struct<a: int64, c: int64>)
> {code}
> (this is still OK, but if you had a ChunkedArray, it certainly gets annoying)
> One option could be to expand the existing {{struct_field}} to allow selecting multiple fields (although that probably gets ambigous/confusing with how you currently select a recursively nested field -> [0, 1] currently means "first child, second subchild" and not "first and second child"). 
> Or a new kernel like "struct_subset" or some other name.
> This might also overlap with general projection functionality? (cc [~westonpace])



--
This message was sent by Atlassian Jira
(v8.20.1#820001)