[GitHub] [arrow-rs] tustvold commented on issue #1269: Provide an `async` ParquetWriter for arrow

["tustvold (via GitHub)" <gi...@apache.org>]

tustvold commented on issue #1269:
URL: https://github.com/apache/arrow-rs/issues/1269#issuecomment-1469561512

   I wonder if this could simply be implemented by adding an `inner_mut` method to [ArrowWriter](https://docs.rs/parquet/latest/parquet/arrow/arrow_writer/struct.ArrowWriter.html). 
   
   This would then allow providing a `Vec<u8>` as the writer, and then periodically gaining access to it and flushing its contents asynchronously. We could provide an AsyncWriter that encapsulates this logic,  but we could also just provide a code example in a doc comment.
   
   The nature of parquet is that an entire row group is buffered up and written in one shot, as data for different columns cannot be interleaved, so I'm not sure it is possible to do better than this


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org