You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Micah Kornfield (Jira)" <ji...@apache.org> on 2022/03/28 17:48:00 UTC
[jira] [Comment Edited] (ARROW-16048) [PyArrow] Null buffers with Pickle protocol.
[ https://issues.apache.org/jira/browse/ARROW-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513531#comment-17513531 ]
Micah Kornfield edited comment on ARROW-16048 at 3/28/22, 5:47 PM:
-------------------------------------------------------------------
I don't think this affect faithfulness. Another option specifically for pickle could be to modify the condition in [__reduce_ex|https://github.com/apache/arrow/blob/d327f694d7cd6b5919fe1df7d9dea6e7ebef03e1/python/pyarrow/io.pxi#L1119] which returns pybytes also when the underlying buffer has size 0?
was (Author: emkornfield):
I don't think this affect faithfulness. Another option specifically for pickle could be to modify the condition in [__reduce_ex_]([https://github.com/apache/arrow/blob/d327f694d7cd6b5919fe1df7d9dea6e7ebef03e1/python/pyarrow/io.pxi#L1119)] which returns pybytes also when the underlying buffer has size 0?
> [PyArrow] Null buffers with Pickle protocol.
> --------------------------------------------
>
> Key: ARROW-16048
> URL: https://issues.apache.org/jira/browse/ARROW-16048
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Micah Kornfield
> Assignee: Micah Kornfield
> Priority: Minor
>
> When underlying buffers are null they populate the buffer protocol ".buf" value with a null value. In some cases this can violate contracts [asserted in cpython|https://github.com/python/cpython/blob/882d8096c262a5945e0cfdd706e5db3ad2b73543/Modules/_pickle.c#L1072]. It might be best to always return an empty non-null buffer when the underlying buffer is null.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)