You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/02/13 12:11:02 UTC

[GitHub] [arrow] alamb commented on pull request #9485: ARROW-11616: [Rust][DataFusion] Add collect_partitioned on DataFrame

alamb commented on pull request #9485:
URL: https://github.com/apache/arrow/pull/9485#issuecomment-778609424


   > Nice improvement. Thanks @seddonm1. At some point we may want collect to return an iterator over batches rather than a vec.
   
   @andygrove  I think we already kind of have this -- namely `SendableRecordBatchStream` (the thing you get from calling `execute` directly is an "Iterator" (well, it is an `async` iterator). To get an actual `Iterator` from the async one, we would probably need to buffer the `RecordBatch`es anyway so I think making that buffering clear to the end user by passing out a `Vec` is a reasonable API. That is my $0.02 anyways


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org