You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "mapleFU (via GitHub)" <gi...@apache.org> on 2025/03/27 08:27:22 UTC
[PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
mapleFU opened a new pull request, #45954:
URL: https://github.com/apache/arrow/pull/45954
### Rationale for this change
### What changes are included in this PR?
### Are these changes tested?
### Are there any user-facing changes?
Bugfix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #45954:
URL: https://github.com/apache/arrow/pull/45954#discussion_r2017132055
##########
cpp/src/arrow/util/async_generator.h:
##########
@@ -800,16 +816,18 @@ class ReadaheadGenerator {
: source_generator(std::move(source_generator)), max_readahead(max_readahead) {}
void MarkFinishedIfDone(const T& next_result) {
+ // ASSERT_HELD(mu)
if (IsIterationEnd(next_result)) {
- finished.store(true);
+ finished = true;
}
}
AsyncGenerator<T> source_generator;
int max_readahead;
Future<> final_future = Future<>::Make();
- std::atomic<int> num_running{0};
- std::atomic<bool> finished{false};
+ int num_running{0}; // GUARDED_BY(mu)
Review Comment:
This is just my code style, also related tool is https://clang.llvm.org/docs/ThreadSafetyAnalysis.html , I just follow same style here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758234354
:warning: GitHub issue #45953 **has no components**, please add labels for components.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #45954:
URL: https://github.com/apache/arrow/pull/45954#discussion_r2017087658
##########
cpp/src/arrow/util/async_generator.h:
##########
@@ -800,16 +816,18 @@ class ReadaheadGenerator {
: source_generator(std::move(source_generator)), max_readahead(max_readahead) {}
void MarkFinishedIfDone(const T& next_result) {
+ // ASSERT_HELD(mu)
if (IsIterationEnd(next_result)) {
- finished.store(true);
+ finished = true;
}
}
AsyncGenerator<T> source_generator;
int max_readahead;
Future<> final_future = Future<>::Make();
- std::atomic<int> num_running{0};
- std::atomic<bool> finished{false};
+ int num_running{0}; // GUARDED_BY(mu)
Review Comment:
Is `GUARDED_BY` an annotation for some kind of static analysis tool? Which one?
##########
cpp/src/arrow/util/async_generator.h:
##########
@@ -800,16 +816,18 @@ class ReadaheadGenerator {
: source_generator(std::move(source_generator)), max_readahead(max_readahead) {}
void MarkFinishedIfDone(const T& next_result) {
+ // ASSERT_HELD(mu)
if (IsIterationEnd(next_result)) {
- finished.store(true);
+ finished = true;
}
}
AsyncGenerator<T> source_generator;
int max_readahead;
Future<> final_future = Future<>::Make();
- std::atomic<int> num_running{0};
- std::atomic<bool> finished{false};
+ int num_running{0}; // GUARDED_BY(mu)
+ bool finished{false}; // GUARDED_BY(mu)
+ arrow::util::Mutex mu;
Review Comment:
Please call this `mutex`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2757175161
:warning: GitHub issue #45953 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758174009
FWIW, it is because of C++/CLI: https://github.com/apache/arrow/issues/45810
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758521289
@pitrou @zanmato1984 would you mind take a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758734578
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #45954:
URL: https://github.com/apache/arrow/pull/45954#discussion_r2020577958
##########
cpp/src/arrow/util/async_generator.h:
##########
@@ -772,20 +782,26 @@ class ReadaheadGenerator {
Future<T> operator()() {
if (state_->readahead_queue.empty()) {
// This is the first request, let's pump the underlying queue
- state_->num_running.store(state_->max_readahead);
+ {
+ auto guard = state_->mutex.Lock();
+ state_->num_running = state_->max_readahead;
+ }
for (int i = 0; i < state_->max_readahead; i++) {
auto next = state_->source_generator();
auto next_after_check = AddMarkFinishedContinuation(std::move(next));
state_->readahead_queue.push(std::move(next_after_check));
}
}
// Pop one and add one
- auto result = state_->readahead_queue.front();
+ auto result = std::move(state_->readahead_queue.front());
state_->readahead_queue.pop();
- if (state_->finished.load()) {
+ auto guard = state_->mutex.Lock();
+ if (state_->finished) {
+ guard.Unlock();
state_->readahead_queue.push(AsyncGeneratorEnd<T>());
} else {
- state_->num_running.fetch_add(1);
+ ++state_->num_running;
+ guard.Unlock();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758158312
@pitrou @zanmato1984 This seems fixed the bug, however, lint disables `<mutex>` in `async_generator.h`. Why is this rule here? Should I use atomic and cas or using other way to bypass this problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "zanmato1984 (via GitHub)" <gi...@apache.org>.
zanmato1984 commented on code in PR #45954:
URL: https://github.com/apache/arrow/pull/45954#discussion_r2020551248
##########
cpp/src/arrow/util/async_generator.h:
##########
@@ -772,20 +782,26 @@ class ReadaheadGenerator {
Future<T> operator()() {
if (state_->readahead_queue.empty()) {
// This is the first request, let's pump the underlying queue
- state_->num_running.store(state_->max_readahead);
+ {
+ auto guard = state_->mutex.Lock();
+ state_->num_running = state_->max_readahead;
+ }
for (int i = 0; i < state_->max_readahead; i++) {
auto next = state_->source_generator();
auto next_after_check = AddMarkFinishedContinuation(std::move(next));
state_->readahead_queue.push(std::move(next_after_check));
}
}
// Pop one and add one
- auto result = state_->readahead_queue.front();
+ auto result = std::move(state_->readahead_queue.front());
state_->readahead_queue.pop();
- if (state_->finished.load()) {
+ auto guard = state_->mutex.Lock();
+ if (state_->finished) {
+ guard.Unlock();
state_->readahead_queue.push(AsyncGeneratorEnd<T>());
} else {
- state_->num_running.fetch_add(1);
+ ++state_->num_running;
+ guard.Unlock();
Review Comment:
Maybe we can make the locking more compact and implicit (explicit unlocking in multiple branches could be somehow error-prone?), like:
```C++
bool finished = false;
{
auto guard = state_->mutex.Lock();
if (finished = state_->finished; !finished) {
++state_->num_running;
}
}
if (finished) {
// ...
} else {
// ...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758750026
Revision: 740b1cfebae3db46247acbe5415ab52d6404e850
Submitted crossbow builds: [ursacomputing/crossbow @ actions-c21700c42d](https://github.com/ursacomputing/crossbow/branches/all?query=actions-c21700c42d)
|Task|Status|
|----|------|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193102/job/39533532514)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193675/job/39533536605)|
|example-cpp-tutorial|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193471/job/39533535040)|
|test-alpine-linux-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193089/job/39533533028)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193496/job/39533535641)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193166/job/39533532670)|
|test-conda-cpp-meson|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193126/job/39533532558)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193344/job/39533534363)|
|test-cuda-cpp-ubuntu-22.04-cuda-11.7.1|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193779/job/39533536960)|
|test-debian-12-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193521/job/39533535671)|
|test-debian-12-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193679/job/39533536568)|
|test-fedora-39-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193674/job/39533536180)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193097/job/39533532569)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193212/job/39533533004)|
|test-ubuntu-22.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193548/job/39533536129)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193660/job/39533536219)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193671/job/39533536177)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193746/job/39533536668)|
|test-ubuntu-24.04-cpp-bundled-offline|[](https://github.com/ursacomputing/crossbow/actions/runs/14112194025/job/39533538265)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193500/job/39533535679)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193476/job/39533535712)|
|test-ubuntu-24.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/14112194662/job/39533539450)|
|test-ubuntu-24.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/14112193690/job/39533536644)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2757313654
sidenote: a `std::atomic<int64_t> num_running{0};` with atomic handling finished and num_running also fixes this case, however, we use lock to prove it's right firstly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "zanmato1984 (via GitHub)" <gi...@apache.org>.
zanmato1984 commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758178130
> @pitrou @zanmato1984 This seems fixed the bug, however, lint disables `<mutex>` in `async_generator.h`. Why is this rule here? Should I use atomic and cas or using other way to bypass this problem?
A related discussion about the <mutex> lint rule and the deprecation of support for C++/CLI can be found here https://github.com/apache/arrow/pull/45674#issuecomment-2726284780.
And it turns out that we are good to get rid of it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU merged PR #45954:
URL: https://github.com/apache/arrow/pull/45954
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2767268218
After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit 19033bc1d0ae88cae8dca0676a69feb0bd1dc05a.
None of the specified runs were found on the Conbench server.
The [full Conbench report](https://github.com/apache/arrow/runs/39724781186) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758240192
:warning: GitHub issue #45953 **has no components**, please add labels for components.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2766407754
Comment fixed, I'll merge this after all ci passes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2757175187
:warning: GitHub issue #45953 **has no components**, please add labels for components.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "zanmato1984 (via GitHub)" <gi...@apache.org>.
zanmato1984 commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758179777
> > Why is this rule here? Should I use atomic and cas or using other way to bypass this problem?
>
>
>
> You can use `arrow/util/mutex.h`
Right, this works too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758172339
> Why is this rule here? Should I use atomic and cas or using other way to bypass this problem?
You can use `arrow/util/mutex.h`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] DNM: Trying to use lock to fix bug in Generator [arrow]
Posted by "EnricoMi (via GitHub)" <gi...@apache.org>.
EnricoMi commented on PR #45954:
URL: https://github.com/apache/arrow/pull/45954#issuecomment-2758062727
@mapleFU I can confirm this fixes the `Future` issue!
Only the `TwoLevelCacheWithExpirationTest.CleanupPeriodOk` occurred across 100 test runs:
https://github.com/EnricoMi/arrow/actions/runs/14102683222/job/39502158145#step:10:972
https://github.com/EnricoMi/arrow/actions/runs/14102683222/job/39502158191#step:12:336
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #45954:
URL: https://github.com/apache/arrow/pull/45954#discussion_r2017132055
##########
cpp/src/arrow/util/async_generator.h:
##########
@@ -800,16 +816,18 @@ class ReadaheadGenerator {
: source_generator(std::move(source_generator)), max_readahead(max_readahead) {}
void MarkFinishedIfDone(const T& next_result) {
+ // ASSERT_HELD(mu)
if (IsIterationEnd(next_result)) {
- finished.store(true);
+ finished = true;
}
}
AsyncGenerator<T> source_generator;
int max_readahead;
Future<> final_future = Future<>::Make();
- std::atomic<int> num_running{0};
- std::atomic<bool> finished{false};
+ int num_running{0}; // GUARDED_BY(mu)
Review Comment:
This is just my code style, related code is in https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-45953: [C++] Use lock to fix atomic bug in ReadaheadGenerator [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #45954:
URL: https://github.com/apache/arrow/pull/45954#discussion_r2021141798
##########
cpp/src/arrow/util/async_generator.h:
##########
@@ -772,20 +782,30 @@ class ReadaheadGenerator {
Future<T> operator()() {
if (state_->readahead_queue.empty()) {
// This is the first request, let's pump the underlying queue
- state_->num_running.store(state_->max_readahead);
+ {
+ auto guard = state_->mutex.Lock();
+ state_->num_running = state_->max_readahead;
+ }
for (int i = 0; i < state_->max_readahead; i++) {
auto next = state_->source_generator();
auto next_after_check = AddMarkFinishedContinuation(std::move(next));
state_->readahead_queue.push(std::move(next_after_check));
}
}
// Pop one and add one
- auto result = state_->readahead_queue.front();
+ auto result = std::move(state_->readahead_queue.front());
state_->readahead_queue.pop();
- if (state_->finished.load()) {
+ bool is_finished = false;
+ {
+ auto guard = state_->mutex.Lock();
+ is_finished = state_->finished;
+ if (!is_finished) {
+ ++state_->num_running;
+ }
Review Comment:
Suggestion: add a comment to explain this
```suggestion
if (!is_finished) {
// We're going to push to the queue below, but we need
// to update `num_running` while we're holding the lock.
++state_->num_running;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org