You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/09/20 14:40:15 UTC

[GitHub] [arrow] lidavidm opened a new pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

lidavidm opened a new pull request #11189:
URL: https://github.com/apache/arrow/pull/11189


   This creates a new concatenating generator which converts AsyncGenerator<optional<vector<T>>> to AsyncGenerator<T> and supports async-reentrancy. When combined with a readahead generator, this enables us to get parallelism in scanning Parquet files.
   
   This also properly chunks each row group into smaller batches as mentioned in ARROW-14024, though we can add more specific tests in that JIRA (and fix the issue for IPC as well).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm closed pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm closed pull request #11189:
URL: https://github.com/apache/arrow/pull/11189


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-924954728


   I think the segfault is the same one as in https://github.com/apache/arrow/pull/11207/commits/b3e61169d621205394d28f2b6e4b515896337e5e which I've (hopefully) fixed in #11207


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-922992268


   Benchmark runs are scheduled for baseline = 7128b5eb4302a768e48e0a4718b445495fc3e107 and contender = dc36f09c16a2933614b41f8912968cf1cfb364d8. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Skipped :warning: Provided benchmark filters do not have any benchmark groups to be executed on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/14909b12ff3b44afb6c2be83853db45b...deb4ec4f85ba4cab97cdc36a4c2d5d9b/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/f7dbbd8836ac43f69a52b17caa64dd26...f0e710cddcef40e89344f39f2dd0e6ba/)
   [Skipped :warning: Only ['C++', 'Java'] langs are supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/2cb2668a5e37451f9d68770ce2a12a09...16fc2e6d8ec346679826ead5c66f42b7/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-933407128


   Benchmark runs are scheduled for baseline = dd6c325afb6f090c9e4fb72ce6068c94857fb7bd and contender = deaab5b53957772a643fcb72fe6669e72637784d. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Skipped :warning: Provided benchmark filters do not have any benchmark groups to be executed on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/f6660218c4d64594b8b96f3d37f37165...4df79e9dddf94ed8b93f69b7e698fb2f/)
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/27344c56dee34c0a9604e3f4734357e9...43d9df0c39554f49b1bcc238a21fd4eb/)
   [Skipped :warning: Only ['C++', 'Java'] langs are supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/9f1f7109916a4b1b8f3a02bb07961581...75ad6e25a3fd460592dbb080b419749a/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-933407128


   Benchmark runs are scheduled for baseline = dd6c325afb6f090c9e4fb72ce6068c94857fb7bd and contender = deaab5b53957772a643fcb72fe6669e72637784d. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Skipped :warning: Provided benchmark filters do not have any benchmark groups to be executed on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/f6660218c4d64594b8b96f3d37f37165...4df79e9dddf94ed8b93f69b7e698fb2f/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/27344c56dee34c0a9604e3f4734357e9...43d9df0c39554f49b1bcc238a21fd4eb/)
   [Skipped :warning: Only ['C++', 'Java'] langs are supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/9f1f7109916a4b1b8f3a02bb07961581...75ad6e25a3fd460592dbb080b419749a/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-923201576


   Though for what it's worth, arrowbench gives rather similar results locally using `as.data.frame(run_benchmark(dataset_taxi_parquet, n_iter=5, cpu_count=c(4)))`.
   
   Before:
   ```
   Total run time: 9.956319 secs
      iteration      process        real start_mem_bytes end_mem_bytes
   1          1  2.979338342 0.785497665      2493300736    3698921472
   2          2  2.886093707 0.766516447      3699429376    3714109440
   3          3  2.920944454 0.776514053      3714109440    3714109440
   4          4  2.877046089 0.766922712      3714109440    3714109440
   5          5  2.896827784 0.770490646      3714109440    3714109440
   6          1 18.316171634 4.632888794      2493300736    3777204224
   7          2 17.657202597 4.614435434      3731472384    3815968768
   8          3 17.561911295 4.556154490      3744194560    3779891200
   9          4 17.775040099 4.648452759      3744194560    3815968768
   10         5 17.586604237 4.563611269      3744194560    3779891200
   11         1  0.964031791 0.582021236      2493296640    3450097664
   12         2  0.771914299 0.414866209      3342176256    3463372800
   13         3  0.770634774 0.410928965      3347357696    3527225344
   14         4  0.798272907 0.439264059      3352600576    3523477504
   15         5  0.795549228 0.436603546      3357843456    3523641344
   16         1  0.488568257 0.068626642      2493296640    3346874368
   17         2  0.001445519 0.001448631      3346874368    3346874368
   18         3  0.001467639 0.001470804      3346874368    3346874368
   19         4  0.001425503 0.001428366      3346874368    3346874368
   20         5  0.001460951 0.001464367      3346874368    3346874368
      max_mem_bytes gc_level0 gc_level1 gc_level2          query cpu_count
   1     3720196096         1         0         0       vignette         4
   2     3720196096         0         0         0       vignette         4
   3     3720196096         0         0         0       vignette         4
   4     3720196096         0         0         0       vignette         4
   5     3720196096         0         0         0       vignette         4
   6     3796340736         2         0         1 payment_type_3         4
   7     3815968768         1         0         1 payment_type_3         4
   8     3815968768         1         0         0 payment_type_3         4
   9     3815968768         0         0         1 payment_type_3         4
   10    3815968768         1         0         0 payment_type_3         4
   11    3481989120         2         1         3 small_no_files         4
   12    3513794560         2         0         1 small_no_files         4
   13    3527225344         1         0         1 small_no_files         4
   14    3527225344         2         0         1 small_no_files         4
   15    3527225344         0         1         1 small_no_files         4
   16    3346874368         0         0         0     count_rows         4
   17    3346874368         0         0         0     count_rows         4
   18    3346874368         0         0         0     count_rows         4
   19    3346874368         0         0         0     count_rows         4
   20    3346874368         0         0         0     count_rows         4
   ```
   
   After:
   ```
   Total run time: 9.995613 secs
      iteration      process        real start_mem_bytes end_mem_bytes
   1          1  3.097148086 0.810223818      2493313024    3759751168
   2          2  2.987976486 0.778924227      3760332800    3779207168
   3          3  2.969162397 0.776125669      3779207168    3779207168
   4          4  2.969802084 0.777535439      3779207168    3779207168
   5          5  2.967313487 0.775563955      3779207168    3779207168
   6          1 18.042212503 4.552006245      2493313024    3805003776
   7          2 17.391559404 4.544502735      3759271936    3832758272
   8          3 17.281636477 4.486674547      3760984064    3796680704
   9          4 17.363466488 4.545516968      3760984064    3832758272
   10         5 17.310964707 4.487994194      3760984064    3796680704
   11         1  1.036147931 0.602855206      2493313024    3547631616
   12         2  0.797990419 0.419062376      3439710208    3601801216
   13         3  0.793205374 0.414812803      3485786112    3683479552
   14         4  0.801588050 0.428945065      3508854784    3674488832
   15         5  0.834174869 0.443217278      3508854784    3674656768
   16         1  0.500276311 0.069972277      2493321216    3346898944
   17         2  0.001478722 0.001481056      3346898944    3346898944
   18         3  0.001478401 0.001481295      3346898944    3346898944
   19         4  0.001472831 0.001475811      3346898944    3346898944
   20         5  0.001461316 0.001464128      3346898944    3346898944
      max_mem_bytes gc_level0 gc_level1 gc_level2          query cpu_count
   1     3759751168         1         0         0       vignette         4
   2     3779207168         0         0         0       vignette         4
   3     3779207168         0         0         0       vignette         4
   4     3779207168         0         0         0       vignette         4
   5     3779207168         0         0         0       vignette         4
   6     3824140288         2         0         1 payment_type_3         4
   7     3832758272         1         0         1 payment_type_3         4
   8     3832758272         1         0         0 payment_type_3         4
   9     3832758272         0         0         1 payment_type_3         4
   10    3832758272         1         0         0 payment_type_3         4
   11    3579523072         2         1         3 small_no_files         4
   12    3652222976         2         0         1 small_no_files         4
   13    3683479552         1         0         1 small_no_files         4
   14    3683479552         2         0         1 small_no_files         4
   15    3683479552         0         1         1 small_no_files         4
   16    3414007808         0         0         0     count_rows         4
   17    3414007808         0         0         0     count_rows         4
   18    3414007808         0         0         0     count_rows         4
   19    3414007808         0         0         0     count_rows         4
   20    3414007808         0         0         0     count_rows         4
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-933407083


   @ursabot please benchmark name=file-read lang=Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-924887696


   This helps quite a bit on all the [partitioned-dataset-filter](https://conbench.ursa.dev/compare/batches/3d25d19e1b2a46468bd5ddf5e82946e2...e0751834febe4a5c93fecd597faa1986/) benchmarks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-923135823


   It appears this helps a bit in some benchmarks but hurts the taxi-parquet benchmark.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] westonpace commented on a change in pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
westonpace commented on a change in pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#discussion_r712604377



##########
File path: cpp/src/arrow/util/async_generator.h
##########
@@ -1116,6 +1116,151 @@ AsyncGenerator<T> MakeConcatenatedGenerator(AsyncGenerator<AsyncGenerator<T>> so
   return MergedGenerator<T>(std::move(source), 1);
 }
 
+/// \brief See MakeConcatenatedVectorGenerator
+template <typename T>
+class ConcatenatedVectorGenerator {
+ public:
+  explicit ConcatenatedVectorGenerator(
+      AsyncGenerator<util::optional<std::vector<T>>> inner)
+      : state_(std::make_shared<State>(std::move(inner))) {}
+
+  Future<T> operator()() { return state_->Pull(); }
+
+ private:
+  struct State : public std::enable_shared_from_this<State> {
+    explicit State(AsyncGenerator<util::optional<std::vector<T>>> inner)
+        : inner(std::move(inner)) {}
+
+    Future<T> Pull() {
+      auto guard = mutex.Lock();
+      if (finished) {
+        if (!error.ok()) {
+          return std::move(error);
+        }
+        return IterationTraits<T>::End();
+      }
+      if (available.empty()) {
+        auto waiting_future = Future<T>::Make();
+        waiting.push_back(waiting_future);
+
+        if (!waiting_for_results) {
+          waiting_for_results = true;
+          auto fut = inner();
+          // If fut is already complete, this may end up running the callback immediately,
+          // so unlock first
+          guard.Unlock();
+          auto self = this->shared_from_this();
+          fut.AddCallback(
+              [self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+                auto result = fut.MoveResult();
+                self->ProcessResult(std::move(result));

Review comment:
       ```suggestion
                   self->ProcessResult(fut.MoveResult());
   ```
   Can you combine these two lines?  Also, is the `fut` capture just to avoid the result copy?

##########
File path: cpp/src/arrow/dataset/file_parquet.cc
##########
@@ -489,9 +489,13 @@ Result<RecordBatchGenerator> ParquetFileFormat::ScanBatchesAsync(
         auto parquet_scan_options,
         GetFragmentScanOptions<ParquetFragmentScanOptions>(
             kParquetTypeName, options.get(), default_fragment_scan_options));
-    ARROW_ASSIGN_OR_RAISE(auto generator, reader->GetRecordBatchGenerator(
-                                              reader, row_groups, column_projection,
-                                              ::arrow::internal::GetCpuThreadPool()));
+    // Assume 1 row group corresponds to 1 batch (this factor could be
+    // improved by looking at metadata)
+    int row_group_readahead = options->batch_readahead;
+    ARROW_ASSIGN_OR_RAISE(
+        auto generator, reader->GetRecordBatchGenerator(
+                            reader, row_groups, column_projection,
+                            ::arrow::internal::GetCpuThreadPool(), row_group_readahead));
     return MakeReadaheadGenerator(std::move(generator), options->batch_readahead);

Review comment:
       Is this redundant at this point since you apply the row group readahead in the file?

##########
File path: cpp/src/arrow/util/async_generator.h
##########
@@ -1116,6 +1116,151 @@ AsyncGenerator<T> MakeConcatenatedGenerator(AsyncGenerator<AsyncGenerator<T>> so
   return MergedGenerator<T>(std::move(source), 1);
 }
 
+/// \brief See MakeConcatenatedVectorGenerator
+template <typename T>
+class ConcatenatedVectorGenerator {
+ public:
+  explicit ConcatenatedVectorGenerator(
+      AsyncGenerator<util::optional<std::vector<T>>> inner)
+      : state_(std::make_shared<State>(std::move(inner))) {}
+
+  Future<T> operator()() { return state_->Pull(); }
+
+ private:
+  struct State : public std::enable_shared_from_this<State> {
+    explicit State(AsyncGenerator<util::optional<std::vector<T>>> inner)
+        : inner(std::move(inner)) {}
+
+    Future<T> Pull() {
+      auto guard = mutex.Lock();
+      if (finished) {
+        if (!error.ok()) {
+          return std::move(error);
+        }
+        return IterationTraits<T>::End();
+      }
+      if (available.empty()) {
+        auto waiting_future = Future<T>::Make();
+        waiting.push_back(waiting_future);
+
+        if (!waiting_for_results) {
+          waiting_for_results = true;
+          auto fut = inner();
+          // If fut is already complete, this may end up running the callback immediately,
+          // so unlock first
+          guard.Unlock();
+          auto self = this->shared_from_this();
+          fut.AddCallback(
+              [self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+                auto result = fut.MoveResult();
+                self->ProcessResult(std::move(result));
+              });
+        }
+        return waiting_future;
+      }
+      assert(waiting.size() == 0);
+      auto result = Future<T>::MakeFinished(available.front());
+      available.pop_front();
+      return result;
+    }
+
+    void Purge() {
+      while (!waiting.empty()) {
+        waiting.front().MarkFinished(IterationTraits<T>::End());
+        waiting.pop_front();
+      }
+    }
+
+    void ProcessResult(Result<util::optional<std::vector<T>>> maybe_result) {
+      auto lock = mutex.Lock();
+      waiting_for_results = false;
+      // We should never call MarkFinished while holding the lock as a
+      // callback may call back into this generator
+      if (!maybe_result.ok()) {
+        finished = true;
+        if (waiting.empty()) {
+          error = maybe_result.status();
+          lock.Unlock();
+        } else {
+          lock.Unlock();
+          waiting.front().MarkFinished(maybe_result.status());
+          waiting.pop_front();
+        }
+        Purge();
+        return;
+      }
+
+      util::optional<std::vector<T>> result = maybe_result.MoveValueUnsafe();
+      if (!result.has_value()) {
+        finished = true;
+        lock.Unlock();
+        Purge();
+        return;
+      }
+
+      // MarkFinished might run a callback that tries to pull from us,
+      // so just record what needs finishing, and complete the futures
+      // later
+      std::vector<T> delivered = std::move(*result);
+      std::vector<Future<T>> deliver_to;
+      auto it = delivered.begin();
+      while (it != delivered.end() && !waiting.empty()) {
+        deliver_to.push_back(waiting.front());
+        waiting.pop_front();
+        it++;
+      }
+      available.insert(available.end(), std::make_move_iterator(it),
+                       std::make_move_iterator(delivered.end()));
+
+      // If there's still more waiting futures, poll the source again
+      Future<util::optional<std::vector<T>>> fut;
+      waiting_for_results = !waiting.empty();
+      if (waiting_for_results) {
+        fut = inner();
+      }
+
+      lock.Unlock();
+      // Now actually MarkFinished
+      auto next = delivered.begin();
+      for (auto& fut : deliver_to) {
+        fut.MarkFinished(std::move(*next));
+        next++;
+      }
+
+      if (!fut.is_valid()) return;
+      // Must do this outside the lock since if fut is already complete, this callback
+      // will run right away
+      auto self = this->shared_from_this();
+      fut.AddCallback([self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+        auto result = fut.MoveResult();
+        self->ProcessResult(std::move(result));
+      });
+    }
+
+    AsyncGenerator<util::optional<std::vector<T>>> inner;
+    util::Mutex mutex;
+    std::deque<Future<T>> waiting;
+    std::deque<T> available;
+    Status error;
+    // Are we finished polling the source?
+    bool finished = false;
+    // Do we have a request in flight for the source?
+    bool waiting_for_results = false;
+  };
+
+  std::shared_ptr<State> state_;
+};
+
+/// \brief Creates a generator that converts a stream of vector<T> to a stream of T.
+///
+/// This generator is async-reentrant but will never pull from source reentrantly.
+/// Instead, combine with MakeReadaheadGenerator.

Review comment:
       Is this much different than `MakeConcatenatedGenerator(MakeMappedGenerator(generator_of_vectors, MakeVectorGenerator))?`

##########
File path: cpp/src/arrow/util/async_generator.h
##########
@@ -1116,6 +1116,151 @@ AsyncGenerator<T> MakeConcatenatedGenerator(AsyncGenerator<AsyncGenerator<T>> so
   return MergedGenerator<T>(std::move(source), 1);
 }
 
+/// \brief See MakeConcatenatedVectorGenerator
+template <typename T>
+class ConcatenatedVectorGenerator {
+ public:
+  explicit ConcatenatedVectorGenerator(
+      AsyncGenerator<util::optional<std::vector<T>>> inner)
+      : state_(std::make_shared<State>(std::move(inner))) {}
+
+  Future<T> operator()() { return state_->Pull(); }
+
+ private:
+  struct State : public std::enable_shared_from_this<State> {
+    explicit State(AsyncGenerator<util::optional<std::vector<T>>> inner)
+        : inner(std::move(inner)) {}
+
+    Future<T> Pull() {
+      auto guard = mutex.Lock();
+      if (finished) {
+        if (!error.ok()) {
+          return std::move(error);
+        }
+        return IterationTraits<T>::End();
+      }
+      if (available.empty()) {
+        auto waiting_future = Future<T>::Make();
+        waiting.push_back(waiting_future);
+
+        if (!waiting_for_results) {
+          waiting_for_results = true;
+          auto fut = inner();
+          // If fut is already complete, this may end up running the callback immediately,
+          // so unlock first
+          guard.Unlock();
+          auto self = this->shared_from_this();
+          fut.AddCallback(
+              [self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+                auto result = fut.MoveResult();
+                self->ProcessResult(std::move(result));
+              });
+        }
+        return waiting_future;
+      }
+      assert(waiting.size() == 0);
+      auto result = Future<T>::MakeFinished(available.front());
+      available.pop_front();
+      return result;
+    }
+
+    void Purge() {
+      while (!waiting.empty()) {
+        waiting.front().MarkFinished(IterationTraits<T>::End());
+        waiting.pop_front();
+      }
+    }
+
+    void ProcessResult(Result<util::optional<std::vector<T>>> maybe_result) {
+      auto lock = mutex.Lock();
+      waiting_for_results = false;
+      // We should never call MarkFinished while holding the lock as a
+      // callback may call back into this generator
+      if (!maybe_result.ok()) {
+        finished = true;
+        if (waiting.empty()) {

Review comment:
       No harm in having this case but I'm not sure if `waiting` can be empty here.

##########
File path: cpp/src/arrow/util/async_generator_test.cc
##########
@@ -461,6 +461,68 @@ TEST(TestAsyncUtil, Concatenated) {
   AssertAsyncGeneratorMatch(expected, concat);
 }
 
+TEST(TestAsyncUtil, ConcatenatedVector) {

Review comment:
       Given the locks you have it would be good to have at least one parallel test using a readahead generator.

##########
File path: cpp/src/arrow/util/async_generator.h
##########
@@ -1116,6 +1116,151 @@ AsyncGenerator<T> MakeConcatenatedGenerator(AsyncGenerator<AsyncGenerator<T>> so
   return MergedGenerator<T>(std::move(source), 1);
 }
 
+/// \brief See MakeConcatenatedVectorGenerator
+template <typename T>
+class ConcatenatedVectorGenerator {
+ public:
+  explicit ConcatenatedVectorGenerator(
+      AsyncGenerator<util::optional<std::vector<T>>> inner)
+      : state_(std::make_shared<State>(std::move(inner))) {}
+
+  Future<T> operator()() { return state_->Pull(); }
+
+ private:
+  struct State : public std::enable_shared_from_this<State> {
+    explicit State(AsyncGenerator<util::optional<std::vector<T>>> inner)
+        : inner(std::move(inner)) {}
+
+    Future<T> Pull() {
+      auto guard = mutex.Lock();
+      if (finished) {
+        if (!error.ok()) {
+          return std::move(error);
+        }
+        return IterationTraits<T>::End();
+      }
+      if (available.empty()) {
+        auto waiting_future = Future<T>::Make();
+        waiting.push_back(waiting_future);
+
+        if (!waiting_for_results) {
+          waiting_for_results = true;
+          auto fut = inner();
+          // If fut is already complete, this may end up running the callback immediately,
+          // so unlock first
+          guard.Unlock();
+          auto self = this->shared_from_this();
+          fut.AddCallback(
+              [self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+                auto result = fut.MoveResult();
+                self->ProcessResult(std::move(result));
+              });
+        }
+        return waiting_future;
+      }
+      assert(waiting.size() == 0);
+      auto result = Future<T>::MakeFinished(available.front());
+      available.pop_front();
+      return result;
+    }
+
+    void Purge() {
+      while (!waiting.empty()) {
+        waiting.front().MarkFinished(IterationTraits<T>::End());
+        waiting.pop_front();
+      }
+    }
+
+    void ProcessResult(Result<util::optional<std::vector<T>>> maybe_result) {
+      auto lock = mutex.Lock();
+      waiting_for_results = false;
+      // We should never call MarkFinished while holding the lock as a
+      // callback may call back into this generator
+      if (!maybe_result.ok()) {
+        finished = true;
+        if (waiting.empty()) {
+          error = maybe_result.status();
+          lock.Unlock();
+        } else {
+          lock.Unlock();
+          waiting.front().MarkFinished(maybe_result.status());
+          waiting.pop_front();
+        }
+        Purge();
+        return;
+      }
+
+      util::optional<std::vector<T>> result = maybe_result.MoveValueUnsafe();
+      if (!result.has_value()) {
+        finished = true;
+        lock.Unlock();
+        Purge();
+        return;
+      }
+
+      // MarkFinished might run a callback that tries to pull from us,
+      // so just record what needs finishing, and complete the futures
+      // later
+      std::vector<T> delivered = std::move(*result);
+      std::vector<Future<T>> deliver_to;
+      auto it = delivered.begin();
+      while (it != delivered.end() && !waiting.empty()) {
+        deliver_to.push_back(waiting.front());
+        waiting.pop_front();
+        it++;
+      }
+      available.insert(available.end(), std::make_move_iterator(it),
+                       std::make_move_iterator(delivered.end()));
+
+      // If there's still more waiting futures, poll the source again
+      Future<util::optional<std::vector<T>>> fut;
+      waiting_for_results = !waiting.empty();
+      if (waiting_for_results) {
+        fut = inner();
+      }
+
+      lock.Unlock();
+      // Now actually MarkFinished
+      auto next = delivered.begin();
+      for (auto& fut : deliver_to) {
+        fut.MarkFinished(std::move(*next));
+        next++;
+      }
+
+      if (!fut.is_valid()) return;
+      // Must do this outside the lock since if fut is already complete, this callback
+      // will run right away
+      auto self = this->shared_from_this();
+      fut.AddCallback([self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+        auto result = fut.MoveResult();
+        self->ProcessResult(std::move(result));
+      });
+    }
+
+    AsyncGenerator<util::optional<std::vector<T>>> inner;
+    util::Mutex mutex;
+    std::deque<Future<T>> waiting;
+    std::deque<T> available;
+    Status error;
+    // Are we finished polling the source?
+    bool finished = false;
+    // Do we have a request in flight for the source?
+    bool waiting_for_results = false;
+  };
+
+  std::shared_ptr<State> state_;
+};
+
+/// \brief Creates a generator that converts a stream of vector<T> to a stream of T.
+///
+/// This generator is async-reentrant but will never pull from source reentrantly.
+/// Instead, combine with MakeReadaheadGenerator.
+template <typename T>
+AsyncGenerator<T> MakeConcatenatedVectorGenerator(

Review comment:
       Maybe `MakeFlattenGenerator` to match `MakeFlattenIterator`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm closed pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm closed pull request #11189:
URL: https://github.com/apache/arrow/pull/11189


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-926041334


   Moving to draft for now since this needs the same fixes as #11207.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] github-actions[bot] commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-922990928


   https://issues.apache.org/jira/browse/ARROW-14026


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-924300963


   Benchmark runs are scheduled for baseline = 18756c2aa60344f9eb5b896ff2cee4e734b92516 and contender = 301e86505a0b2b27a2d6fa7d446e2aa1a136c9ce. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Skipped :warning: Provided benchmark filters do not have any benchmark groups to be executed on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/0ec8ed37a55c4440bc0f7d3ffdb491f8...4610757ce5e64be5a0237b0f2a037b9b/)
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/f9a453ffd2e14e10a74f3e7c71ddfb77...61f200d18df143259815b85175a790ab/)
   [Skipped :warning: Only ['C++', 'Java'] langs are supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/a9ad001bb40e41808430a974c869499e...7a618af070b149e1bfaec737bf21eb69/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-922991831


   @ursabot please benchmark lang=R


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-924300191


   @ursabot please benchmark lang=R


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-933408349


   
   Supported benchmark command examples:
   
   `@ursabot benchmark help`
   
   To run all benchmarks:
   `@ursabot please benchmark`
   
   To filter benchmarks by language:
   `@ursabot please benchmark lang=Python`
   `@ursabot please benchmark lang=C++`
   `@ursabot please benchmark lang=R`
   `@ursabot please benchmark lang=Java`
   `@ursabot please benchmark lang=JavaScript`
   
   To filter Python and R benchmarks by name:
   `@ursabot please benchmark name=file-write`
   `@ursabot please benchmark name=file-write lang=Python`
   `@ursabot please benchmark name=file-.*`
   
   To filter C++ benchmarks by archery --suite-filter and --benchmark-filter:
   `@ursabot please benchmark command=cpp-micro --suite-filter=arrow-compute-vector-selection-benchmark --benchmark-filter=TakeStringRandomIndicesWithNulls/262144/2 --iterations=3`
   
   For other `command=cpp-micro` options, please see https://github.com/ursacomputing/benchmarks/blob/main/benchmarks/cpp_micro_benchmarks.py
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-927985770


   @westonpace any other comments here?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
ursabot commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-924300963


   Benchmark runs are scheduled for baseline = 18756c2aa60344f9eb5b896ff2cee4e734b92516 and contender = 301e86505a0b2b27a2d6fa7d446e2aa1a136c9ce. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Skipped :warning: Provided benchmark filters do not have any benchmark groups to be executed on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/0ec8ed37a55c4440bc0f7d3ffdb491f8...4610757ce5e64be5a0237b0f2a037b9b/)
   [Scheduled] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/f9a453ffd2e14e10a74f3e7c71ddfb77...61f200d18df143259815b85175a790ab/)
   [Skipped :warning: Only ['C++', 'Java'] langs are supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/a9ad001bb40e41808430a974c869499e...7a618af070b149e1bfaec737bf21eb69/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] pitrou commented on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-933408336


   @ursabot help


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] ursabot edited a comment on pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
ursabot edited a comment on pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#issuecomment-922992268


   Benchmark runs are scheduled for baseline = 7128b5eb4302a768e48e0a4718b445495fc3e107 and contender = dc36f09c16a2933614b41f8912968cf1cfb364d8. Results will be available as each benchmark for each run completes.
   Conbench compare runs links:
   [Skipped :warning: Provided benchmark filters do not have any benchmark groups to be executed on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/14909b12ff3b44afb6c2be83853db45b...deb4ec4f85ba4cab97cdc36a4c2d5d9b/)
   [Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/f7dbbd8836ac43f69a52b17caa64dd26...f0e710cddcef40e89344f39f2dd0e6ba/)
   [Skipped :warning: Only ['C++', 'Java'] langs are supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/2cb2668a5e37451f9d68770ce2a12a09...16fc2e6d8ec346679826ead5c66f42b7/)
   Supported benchmarks:
   ursa-i9-9960x: langs = Python, R, JavaScript
   ursa-thinkcentre-m75q: langs = C++, Java
   ec2-t3-xlarge-us-east-2: cloud = True
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#discussion_r712967234



##########
File path: cpp/src/arrow/util/async_generator.h
##########
@@ -1116,6 +1116,151 @@ AsyncGenerator<T> MakeConcatenatedGenerator(AsyncGenerator<AsyncGenerator<T>> so
   return MergedGenerator<T>(std::move(source), 1);
 }
 
+/// \brief See MakeConcatenatedVectorGenerator
+template <typename T>
+class ConcatenatedVectorGenerator {
+ public:
+  explicit ConcatenatedVectorGenerator(
+      AsyncGenerator<util::optional<std::vector<T>>> inner)
+      : state_(std::make_shared<State>(std::move(inner))) {}
+
+  Future<T> operator()() { return state_->Pull(); }
+
+ private:
+  struct State : public std::enable_shared_from_this<State> {
+    explicit State(AsyncGenerator<util::optional<std::vector<T>>> inner)
+        : inner(std::move(inner)) {}
+
+    Future<T> Pull() {
+      auto guard = mutex.Lock();
+      if (finished) {
+        if (!error.ok()) {
+          return std::move(error);
+        }
+        return IterationTraits<T>::End();
+      }
+      if (available.empty()) {
+        auto waiting_future = Future<T>::Make();
+        waiting.push_back(waiting_future);
+
+        if (!waiting_for_results) {
+          waiting_for_results = true;
+          auto fut = inner();
+          // If fut is already complete, this may end up running the callback immediately,
+          // so unlock first
+          guard.Unlock();
+          auto self = this->shared_from_this();
+          fut.AddCallback(
+              [self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+                auto result = fut.MoveResult();
+                self->ProcessResult(std::move(result));
+              });
+        }
+        return waiting_future;
+      }
+      assert(waiting.size() == 0);
+      auto result = Future<T>::MakeFinished(available.front());
+      available.pop_front();
+      return result;
+    }
+
+    void Purge() {
+      while (!waiting.empty()) {
+        waiting.front().MarkFinished(IterationTraits<T>::End());
+        waiting.pop_front();
+      }
+    }
+
+    void ProcessResult(Result<util::optional<std::vector<T>>> maybe_result) {
+      auto lock = mutex.Lock();
+      waiting_for_results = false;
+      // We should never call MarkFinished while holding the lock as a
+      // callback may call back into this generator
+      if (!maybe_result.ok()) {
+        finished = true;
+        if (waiting.empty()) {
+          error = maybe_result.status();
+          lock.Unlock();
+        } else {
+          lock.Unlock();
+          waiting.front().MarkFinished(maybe_result.status());
+          waiting.pop_front();
+        }
+        Purge();
+        return;
+      }
+
+      util::optional<std::vector<T>> result = maybe_result.MoveValueUnsafe();
+      if (!result.has_value()) {
+        finished = true;
+        lock.Unlock();
+        Purge();
+        return;
+      }
+
+      // MarkFinished might run a callback that tries to pull from us,
+      // so just record what needs finishing, and complete the futures
+      // later
+      std::vector<T> delivered = std::move(*result);
+      std::vector<Future<T>> deliver_to;
+      auto it = delivered.begin();
+      while (it != delivered.end() && !waiting.empty()) {
+        deliver_to.push_back(waiting.front());
+        waiting.pop_front();
+        it++;
+      }
+      available.insert(available.end(), std::make_move_iterator(it),
+                       std::make_move_iterator(delivered.end()));
+
+      // If there's still more waiting futures, poll the source again
+      Future<util::optional<std::vector<T>>> fut;
+      waiting_for_results = !waiting.empty();
+      if (waiting_for_results) {
+        fut = inner();
+      }
+
+      lock.Unlock();
+      // Now actually MarkFinished
+      auto next = delivered.begin();
+      for (auto& fut : deliver_to) {
+        fut.MarkFinished(std::move(*next));
+        next++;
+      }
+
+      if (!fut.is_valid()) return;
+      // Must do this outside the lock since if fut is already complete, this callback
+      // will run right away
+      auto self = this->shared_from_this();
+      fut.AddCallback([self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+        auto result = fut.MoveResult();
+        self->ProcessResult(std::move(result));
+      });
+    }
+
+    AsyncGenerator<util::optional<std::vector<T>>> inner;
+    util::Mutex mutex;
+    std::deque<Future<T>> waiting;
+    std::deque<T> available;
+    Status error;
+    // Are we finished polling the source?
+    bool finished = false;
+    // Do we have a request in flight for the source?
+    bool waiting_for_results = false;
+  };
+
+  std::shared_ptr<State> state_;
+};
+
+/// \brief Creates a generator that converts a stream of vector<T> to a stream of T.
+///
+/// This generator is async-reentrant but will never pull from source reentrantly.
+/// Instead, combine with MakeReadaheadGenerator.

Review comment:
       Actually, now that you point that out…I suppose there's one layer of indirection that this gets rid of (once the next vector arrives, any pending futures will get fulfilled right away, instead of going through the callbacks. But I'll check if this actually affects performance/make sure we get parallelism, if not, we should just use the existing ones.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] lidavidm commented on a change in pull request #11189: ARROW-14026: [C++] Enable batch parallelism in Parquet scanner

Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #11189:
URL: https://github.com/apache/arrow/pull/11189#discussion_r713334535



##########
File path: cpp/src/arrow/util/async_generator.h
##########
@@ -1116,6 +1116,151 @@ AsyncGenerator<T> MakeConcatenatedGenerator(AsyncGenerator<AsyncGenerator<T>> so
   return MergedGenerator<T>(std::move(source), 1);
 }
 
+/// \brief See MakeConcatenatedVectorGenerator
+template <typename T>
+class ConcatenatedVectorGenerator {
+ public:
+  explicit ConcatenatedVectorGenerator(
+      AsyncGenerator<util::optional<std::vector<T>>> inner)
+      : state_(std::make_shared<State>(std::move(inner))) {}
+
+  Future<T> operator()() { return state_->Pull(); }
+
+ private:
+  struct State : public std::enable_shared_from_this<State> {
+    explicit State(AsyncGenerator<util::optional<std::vector<T>>> inner)
+        : inner(std::move(inner)) {}
+
+    Future<T> Pull() {
+      auto guard = mutex.Lock();
+      if (finished) {
+        if (!error.ok()) {
+          return std::move(error);
+        }
+        return IterationTraits<T>::End();
+      }
+      if (available.empty()) {
+        auto waiting_future = Future<T>::Make();
+        waiting.push_back(waiting_future);
+
+        if (!waiting_for_results) {
+          waiting_for_results = true;
+          auto fut = inner();
+          // If fut is already complete, this may end up running the callback immediately,
+          // so unlock first
+          guard.Unlock();
+          auto self = this->shared_from_this();
+          fut.AddCallback(
+              [self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+                auto result = fut.MoveResult();
+                self->ProcessResult(std::move(result));
+              });
+        }
+        return waiting_future;
+      }
+      assert(waiting.size() == 0);
+      auto result = Future<T>::MakeFinished(available.front());
+      available.pop_front();
+      return result;
+    }
+
+    void Purge() {
+      while (!waiting.empty()) {
+        waiting.front().MarkFinished(IterationTraits<T>::End());
+        waiting.pop_front();
+      }
+    }
+
+    void ProcessResult(Result<util::optional<std::vector<T>>> maybe_result) {
+      auto lock = mutex.Lock();
+      waiting_for_results = false;
+      // We should never call MarkFinished while holding the lock as a
+      // callback may call back into this generator
+      if (!maybe_result.ok()) {
+        finished = true;
+        if (waiting.empty()) {
+          error = maybe_result.status();
+          lock.Unlock();
+        } else {
+          lock.Unlock();
+          waiting.front().MarkFinished(maybe_result.status());
+          waiting.pop_front();
+        }
+        Purge();
+        return;
+      }
+
+      util::optional<std::vector<T>> result = maybe_result.MoveValueUnsafe();
+      if (!result.has_value()) {
+        finished = true;
+        lock.Unlock();
+        Purge();
+        return;
+      }
+
+      // MarkFinished might run a callback that tries to pull from us,
+      // so just record what needs finishing, and complete the futures
+      // later
+      std::vector<T> delivered = std::move(*result);
+      std::vector<Future<T>> deliver_to;
+      auto it = delivered.begin();
+      while (it != delivered.end() && !waiting.empty()) {
+        deliver_to.push_back(waiting.front());
+        waiting.pop_front();
+        it++;
+      }
+      available.insert(available.end(), std::make_move_iterator(it),
+                       std::make_move_iterator(delivered.end()));
+
+      // If there's still more waiting futures, poll the source again
+      Future<util::optional<std::vector<T>>> fut;
+      waiting_for_results = !waiting.empty();
+      if (waiting_for_results) {
+        fut = inner();
+      }
+
+      lock.Unlock();
+      // Now actually MarkFinished
+      auto next = delivered.begin();
+      for (auto& fut : deliver_to) {
+        fut.MarkFinished(std::move(*next));
+        next++;
+      }
+
+      if (!fut.is_valid()) return;
+      // Must do this outside the lock since if fut is already complete, this callback
+      // will run right away
+      auto self = this->shared_from_this();
+      fut.AddCallback([self, fut](const Result<util::optional<std::vector<T>>>&) mutable {
+        auto result = fut.MoveResult();
+        self->ProcessResult(std::move(result));
+      });
+    }
+
+    AsyncGenerator<util::optional<std::vector<T>>> inner;
+    util::Mutex mutex;
+    std::deque<Future<T>> waiting;
+    std::deque<T> available;
+    Status error;
+    // Are we finished polling the source?
+    bool finished = false;
+    // Do we have a request in flight for the source?
+    bool waiting_for_results = false;
+  };
+
+  std::shared_ptr<State> state_;
+};
+
+/// \brief Creates a generator that converts a stream of vector<T> to a stream of T.
+///
+/// This generator is async-reentrant but will never pull from source reentrantly.
+/// Instead, combine with MakeReadaheadGenerator.

Review comment:
       Alright, it seems to be about the same, so I went with that solution - saves us both work! (This is effectively what you commented in JIRA: we just implement batch readahead as row group readahead.)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org