You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2022/03/28 16:32:00 UTC

[jira] [Commented] (ARROW-16048) [PyArrow] Null buffers with Pickle protocol.

    [ https://issues.apache.org/jira/browse/ARROW-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513483#comment-17513483 ] 

Antoine Pitrou commented on ARROW-16048:
----------------------------------------

Hmm, best would perhaps be to have a dedicated {{NullBuffer}} class so that they can be recreated faithfully when unpickling?

> [PyArrow] Null buffers with Pickle protocol.
> --------------------------------------------
>
>                 Key: ARROW-16048
>                 URL: https://issues.apache.org/jira/browse/ARROW-16048
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Micah Kornfield
>            Assignee: Micah Kornfield
>            Priority: Minor
>
> When underlying buffers are null they populate the buffer protocol ".buf" value with a null value.  In some cases this can violate contracts [asserted in cpython|https://github.com/python/cpython/blob/882d8096c262a5945e0cfdd706e5db3ad2b73543/Modules/_pickle.c#L1072].  It might be best to always return an empty non-null buffer when the underlying buffer is null.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)