[jira] [Updated] (ARROW-1654) [Python] pa.DataType cannot be pickled

["Antoine Pitrou (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Antoine Pitrou updated ARROW-1654:
----------------------------------
    Component/s: Python

> [Python] pa.DataType cannot be pickled
> --------------------------------------
>
>                 Key: ARROW-1654
>                 URL: https://issues.apache.org/jira/browse/ARROW-1654
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Python
>            Reporter: Li Jin
>            Assignee: Wes McKinney
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.8.0
>
>
> In [26]: t
> Out[26]: DataType(int64)
> In [25]: pickle.dumps(t)
> ---------------------------------------------------------------------------
> TypeError                                 Traceback (most recent call last)
> <ipython-input-25-f90063f6658b> in <module>()
> ----> 1 pickle.dumps(t)
> /home/icexelloss/miniconda3/envs/spark-dev/lib/python3.5/site-packages/pyarrow/lib.cpython-35m-x86_64-linux-gnu.so in pyarrow.lib.DataType.__reduce_cython__()
> TypeError: no default __reduce__ due to non-trivial __cinit__
> This is discovered when trying to send a pa.DataType along with a udf in pyspark. The workaround is to send pyspark DataType and convert to pa.DataType. It would be nice to able to pickle pa.DataType.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)