[GitHub] [arrow] lidavidm commented on a change in pull request #10268: ARROW-12687: [C++][Python][Dataset] Convert Scanner into a RecordBatchReader

[GitBox <gi...@apache.org>]

lidavidm commented on a change in pull request #10268:
URL: https://github.com/apache/arrow/pull/10268#discussion_r628462966



##########
File path: cpp/src/arrow/dataset/scanner.cc
##########
@@ -164,6 +164,40 @@ Result<int64_t> Scanner::CountRows() {
   return count;
 }
 
+namespace {
+class ScannerRecordBatchReader : public RecordBatchReader {
+ public:
+  explicit ScannerRecordBatchReader(std::shared_ptr<Schema> schema,
+                                    std::shared_ptr<Scanner> scanner,
+                                    TaggedRecordBatchIterator delegate)
+      : schema_(std::move(schema)),
+        scanner_(std::move(scanner)),
+        delegate_(std::move(delegate)) {}
+
+  std::shared_ptr<Schema> schema() const override { return schema_; }
+  Status ReadNext(std::shared_ptr<RecordBatch>* batch) override {
+    ARROW_ASSIGN_OR_RAISE(auto next, delegate_.Next());
+    if (IsIterationEnd(next)) {
+      *batch = nullptr;
+    } else {
+      *batch = std::move(next.record_batch);
+    }
+    return Status::OK();
+  }
+
+ private:
+  std::shared_ptr<Schema> schema_;
+  std::shared_ptr<Scanner> scanner_;

Review comment:
       I added this on reflex; I've removed it.

##########
File path: python/pyarrow/_dataset.pyx
##########
@@ -2928,6 +2928,13 @@ cdef class Scanner(_Weakrefable):
             result = self.scanner.CountRows()
         return GetResultValue(result)
 
+    def to_reader(self):
+        """Consume this scanner as a RecordBatchReader."""

Review comment:
       In this case, to call RecordBatchReader._export_to_c.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org