You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Partha Dutta <pa...@gmail.com> on 2022/08/01 18:28:53 UTC

[pyarrow] How to properly convert MapArray to StructArray?

I have a need to convert a MapArray type into a StructArray with separate
elements for keys and values. I tried to extract the array from the table
by column and use the corresponding chenks. What I ended up with are two
arrays that are larger than the number of records in the table, so I can
not replace the field.

Is there a way to do this, short of iterating over each MapArray element
one by one?

-- 
Partha Dutta
partha.dutta@gmail.com

Re: [pyarrow] How to properly convert MapArray to StructArray?

Posted by Will Jones <wi...@voltrondata.com>.
Hi Partha, I think this should do what you are looking for:

import pyarrow as pa


ty = pa.map_(pa.int32(), pa.float64())


out_ty = pa.struct([("keys", pa.list_(pa.int32())), ("values",
pa.list_(pa.float64()))])


arr = pa.array([[(1, 1.0), (2, 2.0)], [(3, 3.0)]], type=ty)


res = pa.StructArray.from_arrays(

[pa.ListArray.from_arrays(arr.offsets, arr.keys),
pa.ListArray.from_arrays(arr.offsets, arr.items)],

names=["keys", "values"]

)


assert res.type == out_ty



On Mon, Aug 1, 2022 at 11:29 AM Partha Dutta <pa...@gmail.com> wrote:

> I have a need to convert a MapArray type into a StructArray with separate
> elements for keys and values. I tried to extract the array from the table
> by column and use the corresponding chenks. What I ended up with are two
> arrays that are larger than the number of records in the table, so I can
> not replace the field.
>
> Is there a way to do this, short of iterating over each MapArray element
> one by one?
>
> --
> Partha Dutta
> partha.dutta@gmail.com
>