[jira] [Created] (ARROW-3192) [Java] Implement "ArrowBufReadChannel" abstraction and alternate MessageSerializer that uses this

["Wes McKinney (JIRA)" <ji...@apache.org>]

Wes McKinney created ARROW-3192:
-----------------------------------

             Summary: [Java] Implement "ArrowBufReadChannel" abstraction and alternate MessageSerializer that uses this
                 Key: ARROW-3192
                 URL: https://issues.apache.org/jira/browse/ARROW-3192
             Project: Apache Arrow
          Issue Type: Improvement
          Components: Java
            Reporter: Wes McKinney
             Fix For: 0.12.0


The current MessageSerializer imlementation is wasteful when used to read an IPC payload that is already in-memory in an {{ArrowBuf}}. In particular, reads out of a {{ReadChannel}} require memory allocation

* https://github.com/apache/arrow/blob/master/java/vector/src/main/java/org/apache/arrow/vector/ipc/message/MessageSerializer.java#L569

* https://github.com/apache/arrow/blob/master/java/vector/src/main/java/org/apache/arrow/vector/ipc/message/MessageSerializer.java#L290

In C++, we have abstracted memory allocation out of the IPC read path so that zero-copy is possible. I suggest that a similar mechanism can be developed for Java to improve deserialization performance for in-memory messages. The new interface would return {{ArrowBuf}} when performing reads, which could be zero-copy when possible, but when not the current strategy of allocate-copy could be used



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)