You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2025/10/30 16:03:14 UTC
[PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
pitrou opened a new pull request, #48005:
URL: https://github.com/apache/arrow/pull/48005
### Rationale for this change
The benchmark was instantiating the `ColumnReader` with less values than it would later attempt to read. The `ReadBatch` method would then return 0 prematurely and the loop would never progress.
### Are these changes tested?
Manually and by continuous benchmarking.
### Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
Posted by "voltrondatabot (via GitHub)" <gi...@apache.org>.
voltrondatabot commented on PR #48005:
URL: https://github.com/apache/arrow/pull/48005#issuecomment-3468771455
Benchmark runs are scheduled for commit 2bba103bb26f1e7dc6290f1949c56540e0dd8d8c. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #48005:
URL: https://github.com/apache/arrow/pull/48005#issuecomment-3469437335
Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit 2bba103bb26f1e7dc6290f1949c56540e0dd8d8c.
There were 80 benchmark results indicating a performance regression:
- Pull Request Run on `amd64-c6a-4xlarge-linux` at [2025-10-30 17:09:32Z](https://conbench.ursa.dev/compare/runs/88ef5edded0b4e39b3270ee6758be374...28fa2212774942be9d28a8ef74513379/)
- [`BenchmarkTemporalBinary` (C++) with params=<DayTimeBetween, zoned>/524288/100, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmark](https://conbench.ursa.dev/compare/benchmarks/06901fda4e7c769180002d55f1cacc8a...069039c46d167ceb80008a52cc374956)
- [`BenchmarkTemporalBinary` (C++) with params=<QuartersBetween, zoned>/524288/100, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmark](https://conbench.ursa.dev/compare/benchmarks/06901fda81be744880000a10e778c676...069039c4938976e68000c46c2b7f2a4e)
- and 78 more (see the report linked below)
The [full Conbench report](https://github.com/apache/arrow/runs/54115551158) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #48005:
URL: https://github.com/apache/arrow/pull/48005
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #48005:
URL: https://github.com/apache/arrow/pull/48005#issuecomment-3514124748
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit c6aad8dbde7294f6bbc2d25cfb4b625d8fa467fd.
There weren't enough matching historic benchmark results to make a call on whether there were regressions.
The [full Conbench report](https://github.com/apache/arrow/runs/55026405389) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #48005:
URL: https://github.com/apache/arrow/pull/48005#issuecomment-3468770654
@ursabot please benchmark lang=C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #48005:
URL: https://github.com/apache/arrow/pull/48005#discussion_r2484865021
##########
cpp/src/parquet/column_io_benchmark.cc:
##########
@@ -196,16 +199,17 @@ static void BM_ReadInt64Column(::benchmark::State& state, Repetition::type repet
int64_t stream_size = src->size();
int64_t data_size = int64_values.length() * sizeof(int64_t);
- std::vector<int64_t> values_out(state.range(1));
- std::vector<int16_t> definition_levels_out(state.range(1));
- std::vector<int16_t> repetition_levels_out(state.range(1));
+ std::vector<int64_t> values_out(kBatchSize);
+ std::vector<int16_t> definition_levels_out(kBatchSize);
+ std::vector<int16_t> repetition_levels_out(kBatchSize);
while (state.KeepRunning()) {
std::shared_ptr<Int64Reader> reader =
- BuildReader(src, state.range(1), codec, schema.get());
+ BuildReader(src, kNumValues, codec, schema.get());
Review Comment:
Good catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48004: [C++][Parquet] Fix hang in ColumnReader benchmark [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #48005:
URL: https://github.com/apache/arrow/pull/48005#discussion_r2478681159
##########
cpp/src/parquet/column_io_benchmark.cc:
##########
@@ -196,16 +199,17 @@ static void BM_ReadInt64Column(::benchmark::State& state, Repetition::type repet
int64_t stream_size = src->size();
int64_t data_size = int64_values.length() * sizeof(int64_t);
- std::vector<int64_t> values_out(state.range(1));
- std::vector<int16_t> definition_levels_out(state.range(1));
- std::vector<int16_t> repetition_levels_out(state.range(1));
+ std::vector<int64_t> values_out(kBatchSize);
+ std::vector<int16_t> definition_levels_out(kBatchSize);
+ std::vector<int16_t> repetition_levels_out(kBatchSize);
while (state.KeepRunning()) {
std::shared_ptr<Int64Reader> reader =
- BuildReader(src, state.range(1), codec, schema.get());
+ BuildReader(src, kNumValues, codec, schema.get());
Review Comment:
This is the fix: we were instantiating the ColumnReader with the read batch size (`state.range(1)`) instead of the total number of values (`state.range(0)`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org