[GitHub] [arrow-rs] tustvold opened a new issue #1207: Async IPC Reader/Writer

[GitBox <gi...@apache.org>]

tustvold opened a new issue #1207:
URL: https://github.com/apache/arrow-rs/issues/1207


   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   
   The current IPCReader and IPCWriter require a sync Reader/Writer complicating their use in async systems such as DataFusion. Once such use case can be found in https://github.com/apache/arrow-datafusion/pull/1596 where it is used for spilling buffers
   
   **Describe the solution you'd like**
   
   The encoding/decoding logic to/from Flatbuffers is already largely broken out, it should be a relatively straightforward task of writing an async muxer for this. Ultimately I'm not aware of a way to avoid reading/writing the flatbuffer to an intermediate buffer first, and so I don't think it is possible to do better than this - e.g. by having the codecs read/write directly to/from the file.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org