You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@arrow.apache.org by "Eric Erhardt (JIRA)" <ji...@apache.org> on 2019/02/08 00:19:00 UTC

[jira] [Created] (ARROW-4502) [C#] Add support for zero-copy reads

Eric Erhardt created ARROW-4502:
-----------------------------------

             Summary: [C#] Add support for zero-copy reads
                 Key: ARROW-4502
                 URL: https://issues.apache.org/jira/browse/ARROW-4502
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C#
            Reporter: Eric Erhardt
            Assignee: Eric Erhardt


In the Python (and C++) API, you can create a `RecordBatchStreamReader`, and if you give it an `InputStream` that supports zero-copy reads, you can get back `RecordBatch` objects without allocating new memory and copying all the data.

There is currently no way to read Arrow RecordBatch instances without allocating new memory and copying all the data. We should enable this scenario in the C# API.

 

My proposal is to create a new `class ArrowRecordBatchReader : IArrowReader`. It's constructor will take a `ReadOnlyMemory<byte> data` parameter, and it will be able to read `RecordBatch` instances just like the existing `ArrowStreamReader`. As part of this new class, we will refactor any common code out of `ArrowStreamReader` in order for the parsing logic to be shared, where necessary.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)