You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@arrow.apache.org by "Antoine Pitrou (Jira)" <ji...@apache.org> on 2022/03/24 16:41:00 UTC

[jira] [Commented] (ARROW-16020) [Python] Provide access to buffers underlying scalars

    [ https://issues.apache.org/jira/browse/ARROW-16020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511946#comment-17511946 ] 

Antoine Pitrou commented on ARROW-16020:
----------------------------------------

In any case, if you iterate in values in Python one by one, it is going to be slow.... Is your custom format not Arrow-compatible?

> [Python] Provide access to buffers underlying scalars 
> ------------------------------------------------------
>
>                 Key: ARROW-16020
>                 URL: https://issues.apache.org/jira/browse/ARROW-16020
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: Python
>            Reporter: Kyle Kavanagh
>            Priority: Major
>
> I'm building a process to take data from pyarrow Tables and write their data a memory mapped file in a custom format.  Currently, I iterate through the arrow table and must call as_py() only to convert the python value to bytes and write to the memory mapped file.  If the pyarrow scalar API provided a view over the underlying storage, I could simply memcopy the values from the arrow buffer into the mmap buffer.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)