You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/02/23 23:24:04 UTC

[GitHub] [arrow-datafusion] matthewmturner commented on issue #1544: Streaming support for DataFusion

matthewmturner commented on issue #1544:
URL: https://github.com/apache/arrow-datafusion/issues/1544#issuecomment-1049319476


   I think that @yjshen has lead the way with some great examples on designing new features like this with his work on `ObjectStore` and `MemoryManager`.  Below are the two docs he drafted for that which i think worked really well for collecting feedback from the community and memorializing the design.  Given we are are still at a fairly early stage, and if it isnt too much work on your side, do you think you could make something similar?  Personally, I'm less concerned about hitting the timeline and more getting the design / interface / implementation details ironed out.  
   
   ObjectStore
   https://docs.google.com/document/d/1ZEZqvdohrot0ewtTNeaBtqczOIJ1Q0OnX9PqMMxpOF8/edit#heading=h.358nvuimx7yr
   
   MemoryManager et al
   https://docs.google.com/document/d/1BT5HH-2sKq-Jxo51PNE6l9NNd_F-FyyYcyC3SKTnkIA/edit#heading=h.ims7dd49jei1
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org