You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/05 12:13:29 UTC

[GitHub] [arrow] lidavidm commented on pull request #10070: ARROW-12231: [C++][Python][Dataset] Differentiate one-shot datasets

lidavidm commented on pull request #10070:
URL: https://github.com/apache/arrow/pull/10070#issuecomment-832638716


   I think you could argue that Fragment is just `Iterable<RecordBatch>` while Scanner is `Iterator<RecordBatch>`. While usually Scanner is a rewindable (but not random-access) iterator, it only guarantees ForwardIterator. Furthermore it's pretty simple to implement a one-shot scanner by having it wrap a (non-public-API) one-shot fragment (like the one implemented here).
   
   Or put another way, if we limit the one-shotness to the Scanner, then we can hide the odd nonconforming Dataset/Fragment from the public API.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org