[GitHub] [arrow] jorisvandenbossche edited a comment on pull request #7156: ARROW-8074: [C++][Dataset][Python] FileFragments from buffers and NativeFiles

[GitBox <gi...@apache.org>]

jorisvandenbossche edited a comment on pull request #7156:
URL: https://github.com/apache/arrow/pull/7156#issuecomment-644966084


   Taking a step back: wouldn't it be possible to eg "just" allow to create a Fragment from a buffer instead from a file?
   
   In practice, I think we only need to support dealing with buffers when there is a *single* buffer (so not like paths, where you can have multiple paths or a directory etc). And then do we need discovery at all? If we can construct a Fragment backed by a buffer instead of a file path, then you can create a Dataset from that, either with the physical schema of the fragment (no unification is needed if there is only one) or either with a user-specified schema. 
   And in such a case, the factory can focus on file paths only.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org