You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2026/02/16 08:50:02 UTC
[PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
pitrou opened a new pull request, #49296:
URL: https://github.com/apache/arrow/pull/49296
### Rationale for this change
### What changes are included in this PR?
### Are these changes tested?
### Are there any user-facing changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49296:
URL: https://github.com/apache/arrow/pull/49296#issuecomment-3907579773
FTR @AntoinePrv :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49296:
URL: https://github.com/apache/arrow/pull/49296#issuecomment-3950160149
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49296:
URL: https://github.com/apache/arrow/pull/49296#issuecomment-3950157787
> Just back from holiday and sorry for late reply
It's ok, and thank you for reviewing :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #49296:
URL: https://github.com/apache/arrow/pull/49296#discussion_r2845216351
##########
cpp/src/parquet/encoding_test.cc:
##########
@@ -1967,6 +1961,30 @@ TYPED_TEST(TestDeltaBitPackEncoding, DeltaBitPackedSize) {
ASSERT_EQ(encoded->size(), encoded_size);
}
+TYPED_TEST(TestDeltaBitPackEncoding, ZeroDeltaBitWidth) {
+ // Exercise ranges of zero deltas interspersed between ranges of non-zero deltas.
+ // This checks that the zero bit-width optimization in GH-49266 doesn't mess
+ // decoder state.
+ using T = typename TypeParam::c_type;
+
+ // At least the size of a block
+ constexpr int kRangeSize = 256;
+
+ std::vector<T> int_values;
+ for (int i = 0; i < kRangeSize; ++i) {
+ int_values.push_back((i * 7) % 11);
+ }
+ // Range of equal values, should emit zero-width deltas
+ for (int i = 0; i < kRangeSize * 2; ++i) {
+ int_values.push_back(42);
+ }
+ // int_values.insert(int_values.end(), /*count=*/kRangeSize * k, /*value=*/42);
Review Comment:
This commented-out line should be removed. It appears to be leftover code from development that was replaced by the explicit loop above.
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #49296:
URL: https://github.com/apache/arrow/pull/49296#discussion_r2845443349
##########
cpp/src/parquet/decoder.cc:
##########
@@ -1618,16 +1618,27 @@ class DeltaBitPackDecoder : public TypedDecoderImpl<DType> {
int values_decode = std::min(values_remaining_current_mini_block_,
static_cast<uint32_t>(max_values - i));
- if (decoder_->GetBatch(delta_bit_width_, buffer + i, values_decode) !=
- values_decode) {
- ParquetException::EofException();
- }
- for (int j = 0; j < values_decode; ++j) {
- // Addition between min_delta, packed int and last_value should be treated as
- // unsigned addition. Overflow is as expected.
- buffer[i + j] = static_cast<UT>(min_delta_) + static_cast<UT>(buffer[i + j]) +
- static_cast<UT>(last_value_);
- last_value_ = buffer[i + j];
+ if (delta_bit_width_ == 0) {
+ // Fast path that avoids a back-to-back dependency between two consecutive
+ // computations: we know all deltas decode to zero. We actually don't
+ // even need to decode them.
Review Comment:
Since the bit width is zero, there are zero bytes to decode, so we don't need to advance it.
(also this is the purpose of the additional test)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #49296:
URL: https://github.com/apache/arrow/pull/49296
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #49296:
URL: https://github.com/apache/arrow/pull/49296#discussion_r2845409505
##########
cpp/src/parquet/decoder.cc:
##########
@@ -1618,16 +1618,27 @@ class DeltaBitPackDecoder : public TypedDecoderImpl<DType> {
int values_decode = std::min(values_remaining_current_mini_block_,
static_cast<uint32_t>(max_values - i));
- if (decoder_->GetBatch(delta_bit_width_, buffer + i, values_decode) !=
- values_decode) {
- ParquetException::EofException();
- }
- for (int j = 0; j < values_decode; ++j) {
- // Addition between min_delta, packed int and last_value should be treated as
- // unsigned addition. Overflow is as expected.
- buffer[i + j] = static_cast<UT>(min_delta_) + static_cast<UT>(buffer[i + j]) +
- static_cast<UT>(last_value_);
- last_value_ = buffer[i + j];
+ if (delta_bit_width_ == 0) {
+ // Fast path that avoids a back-to-back dependency between two consecutive
+ // computations: we know all deltas decode to zero. We actually don't
+ // even need to decode them.
Review Comment:
And also not need to advance decoder? It's a little bit confusing when reading but it's right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49296:
URL: https://github.com/apache/arrow/pull/49296#issuecomment-3950172327
Revision: be8b49a46292dd213a5bfd4fb64ed6c8cceaad21
Submitted crossbow builds: [ursacomputing/crossbow @ actions-b4edb848a9](https://github.com/ursacomputing/crossbow/branches/all?query=actions-b4edb848a9)
|Task|Status|
|----|------|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690062/job/64652759151)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689909/job/64652758957)|
|example-cpp-tutorial|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689507/job/64652757237)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689428/job/64652757257)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690030/job/64652759141)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690243/job/64652759878)|
|test-debian-13-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689397/job/64652757279)|
|test-debian-13-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689653/job/64652758018)|
|test-debian-experimental-cpp-gcc-15|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689727/job/64652758097)|
|test-fedora-42-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690412/job/64652760418)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689714/job/64652757972)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689553/job/64652757424)|
|test-ubuntu-22.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689705/job/64652757922)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690136/job/64652759487)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689404/job/64652757253)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690460/job/64652760766)|
|test-ubuntu-24.04-cpp-bundled-offline|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690391/job/64652760480)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689735/job/64652758204)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/22343689952/job/64652758876)|
|test-ubuntu-24.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690477/job/64652761059)|
|test-ubuntu-24.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/22343690107/job/64652759503)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49266: [C++][Parquet] Optimize delta bit-packed decoding when bit-width = 0 [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #49296:
URL: https://github.com/apache/arrow/pull/49296#issuecomment-3954149006
After merging your PR, Conbench analyzed the 2 benchmarking runs that have been run so far on merge-commit 8b50bb1654d8a0bdbc9b6ff7ceded1a836be5801.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/64731473890) has more details. It also includes information about 5 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org