[jira] [Commented] (ARROW-13151) [Python] Unable to read column of `list>`

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368879#comment-17368879 ] 

Joris Van den Bossche commented on ARROW-13151:
-----------------------------------------------

Reading the file itself seems to work OK:

{code}
In [13]: file.read()
Out[13]: 
pyarrow.Table
root: list<item: struct<addr: struct<that: int64, this: int64>>>
  child 0, item: struct<addr: struct<that: int64, this: int64>>
      child 0, addr: struct<that: int64, this: int64>
          child 0, that: int64
          child 1, this: int64
{code}

But it's the reading of only a specific child field that seems to be failing (I can reproduce the failure). 

I am not fully sure what is currently expected to be supported (whether we support reading a single field of a struct column).  cc [~emkornfield]

Thanks for the report!

> [Python] Unable to read column of `list<struct<struct>>`
> --------------------------------------------------------
>
>                 Key: ARROW-13151
>                 URL: https://issues.apache.org/jira/browse/ARROW-13151
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Parquet, Python
>            Reporter: Angus Hollands
>            Priority: Major
>
> Given the following table
> {code:java}
> data = {"root": [[{"addr": {"this": 3, "that": 3}}]]}
> table = pa.Table.from_pydict(data)
> {code}
> reading the nested column leads to an `pyarrow.lib.ArrowInvalid` error:
> {code}
> pq.write_table(table, "/tmp/table.parquet")
> file = pq.ParquetFile("/tmp/table.parquet")
> array = file.read(["root.list.item.addr.that"])
> {code}
> Traceback:
> {code}
> Traceback (most recent call last):
>   File "....", line 21, in <module>
>     array = file.read(["root.list.item.addr.that"])
>   File "/home/angus/.mambaforge/envs/awkward/lib/python3.9/site-packages/pyarrow/parquet.py", line 383, in read
>     return self.reader.read_all(column_indices=column_indices,
>   File "pyarrow/_parquet.pyx", line 1097, in pyarrow._parquet.ParquetReader.read_all
>   File "pyarrow/error.pxi", line 97, in pyarrow.lib.check_status
> pyarrow.lib.ArrowInvalid: List child array invalid: Invalid: Struct child array #0 does not match type field: struct<that: int64> vs struct<that: int64, this: int64>
> {code}
> It's possible that I don't quite understand this properly - am I doing something wrong?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)