[jira] [Assigned] (ARROW-7647) [C++] JSON reader fails to read arrays with few values

["Ben Kietzman (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-7647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ben Kietzman reassigned ARROW-7647:
-----------------------------------

    Assignee: Ben Kietzman

> [C++] JSON reader fails to read arrays with few values
> ------------------------------------------------------
>
>                 Key: ARROW-7647
>                 URL: https://issues.apache.org/jira/browse/ARROW-7647
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, Python
>    Affects Versions: 0.15.1
>         Environment: Ubuntu Linux 18.04
> Python 3.7.5
>            Reporter: Johan Forsberg
>            Assignee: Ben Kietzman
>            Priority: Major
>              Labels: json
>
> Hi! I'm trying to load some nested JSON data and am running into a problem with arrays. I can reproduce it with a slightly modified example from the documentation:
> {code:python}
> from pyarrow import json
> import pyarrow as pa
> with open("test.json", "w") as f:
>     test_json = """{"a": [1], "b": {"c": true, "d": "1991-02-03"}}
> {"a": [], "b": {"c": false, "d": "2019-04-01"}}
> """
>     f.write(test_json)
> json.read_json("test.json")
> {code}
> Running this code with pyarrow 0.15.1 (I also tried 0.14) gives the following error:
> {code:java}
> Traceback (most recent call last):
>   File "issue.py", line 11, in <module>
>     ccs = json.read_json("test.json")
>   File "pyarrow/_json.pyx", line 195, in pyarrow._json.read_json
>   File "pyarrow/public-api.pxi", line 285, in pyarrow.lib.pyarrow_wrap_table
>   File "pyarrow/error.pxi", line 85, in pyarrow.lib.check_status
> pyarrow.lib.ArrowInvalid: Column 0 named a expected length 2 but got length 1
> {code}
> I've tried various combinations and it seems like the error only appears when the *total* number of elements in all the "a" arrays is less than the number of *rows* in the file. I did not expect there to be any relationship between those things and have found nothing in the documentation about it. Is this intentional? If not, I'd suspect there's some problem in the validation step.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)