[jira] [Closed] (ARROW-7961) pyarrow 0.16.0 cannot deserialize content serialised with < 0.16.0

["Wes McKinney (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-7961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney closed ARROW-7961.
-------------------------------
    Resolution: Won't Fix

These APIs are only intended for ephemeral storage with no inter-version compatibility assertions. This is stated in the docstring for the function "transient storage and transport". 

> pyarrow 0.16.0 cannot deserialize content serialised with < 0.16.0
> ------------------------------------------------------------------
>
>                 Key: ARROW-7961
>                 URL: https://issues.apache.org/jira/browse/ARROW-7961
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.16.0
>         Environment: MacOS, python 3.7
>            Reporter: Rob
>            Priority: Major
>
> Pandas data frame has been serialised into a Redis cache using pyarrow 0.14.x.  After upgrading to 0.16.0 deserialise() fails.  Have upgraded to 0.15.1 and this works with objects serialised with 0.14.x
> {{import pyarrow as pa, redis}}
> {{print(pa.__version__)}}
> {{c = redis.Redis.from_url("redis://127.0.0.1")}}
> {{obj = c.get("breakable")}}
> {{df = pa.deserialize(obj)}}
> {{print(df.head())}}
> {{c.set("breakable", pa.serialize(df).to_buffer().to_pybytes())}}
> When run venv with 0.15.1 installed.  No errors. Version number goes to stdout and no stderror.
> When run in 0.16.0 following error is generated
> {{'0.16.0',}}
> {{ 'Traceback (most recent call last):',}}
> {{ ' File "/tmp/pa.py", line 6, in <module>',}}
> {{ ' df = pa.deserialize(obj)',}}
> {{ ' File "pyarrow/serialization.pxi", line 476, in pyarrow.lib.deserialize',}}
> {{ ' File "pyarrow/serialization.pxi", line 438, in pyarrow.lib.deserialize_from',}}
> {{ ' File "pyarrow/serialization.pxi", line 414, in pyarrow.lib.read_serialized',}}
> {{ ' File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status',}}
> {{ 'OSError: Expected IPC message of type unknown but got unknown']}}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)