You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by ALBERTO Bocchinfuso <al...@hotmail.it> on 2018/04/12 19:13:18 UTC

Pickle data from python

Hello,

I cannot pickle RecordBatches, Buffers etc.

I found Issue 1654 in the issue tracker, that has been solved with pull request 1238. But this looks to apply only to the types listed (schemas, DataTypes, etc.).
When I try to Pickle Buffers etc. I get exactly the same error reported in the issue report.
Is the implementation of the possibility of pickling all the data types of pyarrow (with particular attention to RecordBatches etc.) on the agenda?

Thank you,
Alberto

Re: Pickle data from python

Posted by Antoine Pitrou <an...@python.org>.
There's already https://issues.apache.org/jira/browse/ARROW-1715

As for pickling Buffers, it's a bit more contentious.  Perhaps we should
stick to pickling higher-level types (arrays, batches, etc.).

Regards

Antoine.


Le 13/04/2018 à 03:22, Wes McKinney a écrit :
> hi Alberto,
> 
> If you cannot find a JIRA about pickling RecordBatch objects, could
> you please create one? A patch would be welcome for this; it is
> certainly in scope for the project.
> 
> If you encounter any new problems, please open a bug report.
> 
> Thanks!
> Wes
> 
> On Thu, Apr 12, 2018 at 3:13 PM, ALBERTO Bocchinfuso
> <al...@hotmail.it> wrote:
>> Hello,
>>
>> I cannot pickle RecordBatches, Buffers etc.
>>
>> I found Issue 1654 in the issue tracker, that has been solved with pull request 1238. But this looks to apply only to the types listed (schemas, DataTypes, etc.).
>> When I try to Pickle Buffers etc. I get exactly the same error reported in the issue report.
>> Is the implementation of the possibility of pickling all the data types of pyarrow (with particular attention to RecordBatches etc.) on the agenda?
>>
>> Thank you,
>> Alberto

Re: Pickle data from python

Posted by Wes McKinney <we...@gmail.com>.
hi Alberto,

If you cannot find a JIRA about pickling RecordBatch objects, could
you please create one? A patch would be welcome for this; it is
certainly in scope for the project.

If you encounter any new problems, please open a bug report.

Thanks!
Wes

On Thu, Apr 12, 2018 at 3:13 PM, ALBERTO Bocchinfuso
<al...@hotmail.it> wrote:
> Hello,
>
> I cannot pickle RecordBatches, Buffers etc.
>
> I found Issue 1654 in the issue tracker, that has been solved with pull request 1238. But this looks to apply only to the types listed (schemas, DataTypes, etc.).
> When I try to Pickle Buffers etc. I get exactly the same error reported in the issue report.
> Is the implementation of the possibility of pickling all the data types of pyarrow (with particular attention to RecordBatches etc.) on the agenda?
>
> Thank you,
> Alberto