You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2026/04/22 13:45:15 UTC
[PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
pitrou opened a new pull request, #49838:
URL: https://github.com/apache/arrow/pull/49838
### Rationale for this change
`DeltaLengthByteArrayDecoder::DecodeArrow` and `DeltaByteArrayDecoder::DecodeArrow` both allocate a temporary `std::vector` for the entire range of decoded values.
This generates out-of-memory failures in the Parquet encoding fuzzer as this unbounded allocation bypasses the custom fuzzing memory pool.
This issue was found by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/6634166832922624
### What changes are included in this PR?
Use the given MemoryPool to allocate a temporary buffer, so that potential memory limits are applied.
### Are these changes tested?
Yes, using existing tests.
### Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4296924992
Revision: 39fc02fbd7a0c32156a6877f5d6a238915bdc351
Submitted crossbow builds: [ursacomputing/crossbow @ actions-2ca183efe9](https://github.com/ursacomputing/crossbow/branches/all?query=actions-2ca183efe9)
|Task|Status|
|----|------|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062959/job/72519319205)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions/runs/24783063974/job/72519323389)|
|example-cpp-tutorial|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062316/job/72519317371)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062818/job/72519318721)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24783063339/job/72519320465)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/24783063444/job/72519321558)|
|test-debian-13-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/24783064199/job/72519323874)|
|test-debian-13-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/24783063520/job/72519321828)|
|test-debian-experimental-cpp-gcc-15|[](https://github.com/ursacomputing/crossbow/actions/runs/24783063868/job/72519322763)|
|test-fedora-42-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24783064459/job/72519324609)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062335/job/72519317469)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062495/job/72519317583)|
|test-ubuntu-22.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/24783064729/job/72519325623)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/24783064077/job/72519323546)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/24783065020/job/72519326814)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062902/job/72519319307)|
|test-ubuntu-24.04-cpp-bundled-offline|[](https://github.com/ursacomputing/crossbow/actions/runs/24783063209/job/72519320340)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/24783063345/job/72519322554)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/24783064115/job/72519323932)|
|test-ubuntu-24.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062500/job/72519317546)|
|test-ubuntu-24.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/24783062353/job/72519317266)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4306156677
Hi, I also wonder does `DeltaByteArrayFLBADecoder::Decode` matters?
Maybe when I writing this I think this would be small vector, but in fuzz it could be wrongly large...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4303071125
Hi @mapleFU and thanks a lot for the review! :wave:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #49838:
URL: https://github.com/apache/arrow/pull/49838
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4296900239
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4297478354
Revision: 14d3f538edda5d29d0498983e3cb8d98a679defb
Submitted crossbow builds: [ursacomputing/crossbow @ actions-3c858e4ad0](https://github.com/ursacomputing/crossbow/branches/all?query=actions-3c858e4ad0)
|Task|Status|
|----|------|
|example-cpp-minimal-build-static|[](https://github.com/ursacomputing/crossbow/actions/runs/24786453665/job/72531692506)|
|example-cpp-minimal-build-static-system-dependency|[](https://github.com/ursacomputing/crossbow/actions/runs/24786453380/job/72531691390)|
|example-cpp-tutorial|[](https://github.com/ursacomputing/crossbow/actions/runs/24786454365/job/72531695195)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/24786454449/job/72531695057)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24786453176/job/72531690568)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/24786453463/job/72531692882)|
|test-debian-13-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/24786453180/job/72531690596)|
|test-debian-13-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/24786454554/job/72531695349)|
|test-debian-experimental-cpp-gcc-15|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452205/job/72531687274)|
|test-fedora-42-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24786454113/job/72531693883)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452212/job/72531687323)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452249/job/72531687126)|
|test-ubuntu-22.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452315/job/72531687272)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452661/job/72531689091)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/24786454158/job/72531693962)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452858/job/72531691339)|
|test-ubuntu-24.04-cpp-bundled-offline|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452195/job/72531687166)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/24786453863/job/72531692927)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/24786453897/job/72531693062)|
|test-ubuntu-24.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/24786452161/job/72531687207)|
|test-ubuntu-24.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/24786454143/job/72531694799)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4388119742
> Hi, I also wonder does `DeltaByteArrayFLBADecoder::Decode` matters?
>
> Maybe when I writing this I think this would be small vector, but in fuzz it could be wrongly large...
That's a good question. We'll see if the fuzzer flags it, IMHO.
(in the real world I don't think DELTA_BYTE_ARRAY is very interesting for FLBA, so it's probably not widely used?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4305738858
After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit 487f1480914d0066a96db9c8f1395e4e9bdd6b56.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/72726312321) has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #49838:
URL: https://github.com/apache/arrow/pull/49838#issuecomment-4297458369
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-49837: [C++][Parquet] Avoid unbounded temporary std::vector in DELTA_(LENGTH_)BYTE_ARRAY decoder [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #49838:
URL: https://github.com/apache/arrow/pull/49838#discussion_r3125298025
##########
cpp/src/arrow/util/logging_test.cc:
##########
@@ -92,6 +93,9 @@ TEST(ArrowCheck, PayloadEvaluatedOnFailure) {
}
TEST(ArrowLog, MultiThreadedLogging) {
+#ifndef ARROW_ENABLE_THREADING
+ GTEST_SKIP() << "Test requires threading support";
+#endif
Review Comment:
This is a quick fix for an unrelated CI regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org