[jira] [Closed] (ARROW-11371) [Python] Serializing deeply nested structures raises recursion error

["Antoine Pitrou (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-11371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Antoine Pitrou closed ARROW-11371.
----------------------------------
    Resolution: Won't Fix

PyArrow serialization is deprecated, closing as won't fix.

> [Python] Serializing deeply nested structures raises recursion error
> --------------------------------------------------------------------
>
>                 Key: ARROW-11371
>                 URL: https://issues.apache.org/jira/browse/ARROW-11371
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 1.0.1
>         Environment: $ python --version
> Python 3.6.12
> $ pip list | grep pyarrow
> pyarrow                 1.0.1
>            Reporter: Alexander Dmitriev
>            Priority: Minor
>              Labels: pyarrow-serialization
>         Attachments: pyarrow_recursion.py
>
>
> When serializing deeply nested python structures such as:
> {code:java}
> data = [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[None]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]{code}
> or 
> {code:java}
> data = OrderedDict([('a', OrderedDict([('a', OrderedDict([('a', OrderedDict([('a', OrderedDict([('a', OrderedDict([('a', OrderedDict([('a', OrderedDict([('a', OrderedDict([('a', None)]))]))]))]))]))]))]))]))]){code}
> pyarrow raises the following exception:
> {code:java}
> >>> buffer = pyarrow.serialize(data).to_buffer()
> Traceback (most recent call last):
>   File "pyarrow_recursion.py", line 11, in try_recursion
>     buffer = pyarrow.serialize(data).to_buffer()
>   File "pyarrow/serialization.pxi", line 285, in pyarrow.lib.SerializedPyObject.to_buffer
>   File "pyarrow/serialization.pxi", line 248, in pyarrow.lib.SerializedPyObject.total_bytes.__get__
>   File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
> pyarrow.lib.ArrowInvalid: Max recursion depth reached
> {code}
> While this not the prime use case of pyarrow.serialize, it would be nice to have a much larger recursion limit here, to not run into this issue with auto-generated structures quite so often.
> An example script for reproducing the error can be found in the attachments.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)