You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Sebastien Binet (JIRA)" <ji...@apache.org> on 2019/08/02 13:19:00 UTC

[jira] [Commented] (ARROW-6107) [Go] ipc.Writer Option to skip appending data buffers

    [ https://issues.apache.org/jira/browse/ARROW-6107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898880#comment-16898880 ] 

Sebastien Binet commented on ARROW-6107:
----------------------------------------

not saying it wouldn't be advisable nor doable, but: if it's already in a shmem region, why not just use that already?

(and I guess it's kind of implementing: https://issues.apache.org/jira/browse/ARROW-4852)

> [Go] ipc.Writer Option to skip appending data buffers
> -----------------------------------------------------
>
>                 Key: ARROW-6107
>                 URL: https://issues.apache.org/jira/browse/ARROW-6107
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Go
>            Reporter: Nick Poorman
>            Priority: Minor
>
> For cases where we have a known shared memory region, it would be great if the ipc.Writer (and by extension ipc.Reader?) had the ability to write out everything but the actual buffers holding the data. That way we can still utilize the ipc mechanisms to communicate without having to serialize all the underlying data across the wire.
>  
> This seems like it should be possible since the `RecordBatch` flatbuffers only contain the metadata and the underlying data buffers are appended later. We just need to skip appending the underlying data buffers.
>  
> [~sbinet] thoughts?



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)