You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2024/09/04 09:38:15 UTC
[PR] GH-43946: [C++][Parquet] Guard against use of decryptor/encryptor after wipeout [arrow]
pitrou opened a new pull request, #43947:
URL: https://github.com/apache/arrow/pull/43947
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43947:
URL: https://github.com/apache/arrow/pull/43947#discussion_r1744923391
##########
cpp/src/parquet/encryption/encryption_internal.cc:
##########
@@ -413,6 +423,12 @@ class AesDecryptor::AesDecryptorImpl {
}
private:
+ void CheckValid() {
+ if (ctx_ == nullptr) {
Review Comment:
I was under the impression that compilers automatically treated branches that throw an exception to be unlikely, but I can't find a reference. @felipecrv Do you know about that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "adamreeve (via GitHub)" <gi...@apache.org>.
adamreeve commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2330162514
This looks good to me, impressive work tracking down the cause of the segfault!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2328428142
We have a nice example of the guard being triggered here:
https://github.com/ursacomputing/crossbow/actions/runs/10699204038/job/29660230356#step:7:3575
This was a sporadic crash before:
https://github.com/apache/arrow/issues/43057
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43947:
URL: https://github.com/apache/arrow/pull/43947#discussion_r1745036308
##########
cpp/src/parquet/encryption/encryption_internal.cc:
##########
@@ -413,6 +423,12 @@ class AesDecryptor::AesDecryptorImpl {
}
private:
+ void CheckValid() {
+ if (ctx_ == nullptr) {
Review Comment:
Ok, judging from https://www.youtube.com/watch?v=T84swS6DCRo (at 29:00), the gcc compiler already uses such a heuristic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2330968919
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2330974051
Revision: 0fb9ee62dd784491ef3d9de3079cfae2cb0cf6d9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-0c32dece8e](https://github.com/ursacomputing/crossbow/branches/all?query=actions-0c32dece8e)
|Task|Status|
|----|------|
|test-alpine-linux-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135127/job/29716181713)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135391/job/29716183482)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135115/job/29716181703)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135282/job/29716182833)|
|test-cuda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135135/job/29716181681)|
|test-debian-12-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135238/job/29716182636)|
|test-debian-12-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135308/job/29716182747)|
|test-fedora-39-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135214/job/29716182245)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135182/job/29716182169)|
|test-ubuntu-20.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135480/job/29716183808)|
|test-ubuntu-20.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135601/job/29716184285)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135610/job/29716184128)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135703/job/29716184189)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135416/job/29716183529)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135310/job/29716183183)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135468/job/29716183793)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135289/job/29716183048)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135254/job/29716182480)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/10717135178/job/29716182142)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of decryptor/encryptor after wipeout [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2328387646
:warning: GitHub issue #43946 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on code in PR #43947:
URL: https://github.com/apache/arrow/pull/43947#discussion_r1745068112
##########
cpp/src/parquet/encryption/encryption_internal.cc:
##########
@@ -413,6 +423,12 @@ class AesDecryptor::AesDecryptorImpl {
}
private:
+ void CheckValid() {
+ if (ctx_ == nullptr) {
Review Comment:
Good point @mapleFU
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2328396679
@github-actions crossbow submit -g cpp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2330440548
> We have a nice example of the guard being triggered here: https://github.com/ursacomputing/crossbow/actions/runs/10699204038/job/29660230356#step:7:3575
>
> This was a sporadic crash before: #43057
Does it mean that `DatasetEncryptionTest.WriteReadDatasetWithEncryption` test case does not run in the regular CI? Or is it nondeterministic when the decryptor is cleared?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2328402249
Revision: 58b5dafdece1a1308ad1c24bbd67cbf87babb57a
Submitted crossbow builds: [ursacomputing/crossbow @ actions-3ff57908b8](https://github.com/ursacomputing/crossbow/branches/all?query=actions-3ff57908b8)
|Task|Status|
|----|------|
|test-alpine-linux-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203806/job/29660228404)|
|test-build-cpp-fuzz|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203856/job/29660228743)|
|test-conda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10699204017/job/29660230426)|
|test-conda-cpp-valgrind|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203969/job/29660229805)|
|test-cuda-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203871/job/29660228792)|
|test-debian-12-cpp-amd64|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203908/job/29660229495)|
|test-debian-12-cpp-i386|[](https://github.com/ursacomputing/crossbow/actions/runs/10699204070/job/29660230587)|
|test-fedora-39-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203933/job/29660229602)|
|test-ubuntu-20.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203939/job/29660229723)|
|test-ubuntu-20.04-cpp-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/10699204038/job/29660230356)|
|test-ubuntu-20.04-cpp-minimal-with-formats|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203965/job/29660229846)|
|test-ubuntu-20.04-cpp-thread-sanitizer|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203864/job/29660229035)|
|test-ubuntu-22.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203919/job/29660229430)|
|test-ubuntu-22.04-cpp-20|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203882/job/29660228988)|
|test-ubuntu-22.04-cpp-emscripten|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203905/job/29660229445)|
|test-ubuntu-22.04-cpp-no-threading|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203978/job/29660230076)|
|test-ubuntu-24.04-cpp|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203955/job/29660229786)|
|test-ubuntu-24.04-cpp-gcc-13-bundled|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203816/job/29660228383)|
|test-ubuntu-24.04-cpp-gcc-14|[](https://github.com/ursacomputing/crossbow/actions/runs/10699203804/job/29660228369)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2332286520
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 032e6a42bfa359b62d0ed4e5d9b44582a558c87e.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/29742998098) has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on code in PR #43947:
URL: https://github.com/apache/arrow/pull/43947#discussion_r1745043816
##########
cpp/src/parquet/encryption/encryption_internal.cc:
##########
@@ -413,6 +423,12 @@ class AesDecryptor::AesDecryptorImpl {
}
private:
+ void CheckValid() {
+ if (ctx_ == nullptr) {
Review Comment:
CheckValid should be `const`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou merged PR #43947:
URL: https://github.com/apache/arrow/pull/43947
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2330773103
> Does it mean that `DatasetEncryptionTest.WriteReadDatasetWithEncryption` test case does not run in the regular CI? Or is it nondeterministic when the decryptor is cleared?
It seems to be non-deterministic in this particular test, as the dataset reading is multi-threaded (you can see this in the stack traces above).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #43947:
URL: https://github.com/apache/arrow/pull/43947#issuecomment-2328422929
@adamreeve Does this look good to you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-43946: [C++][Parquet] Guard against use of cleared decryptor/encryptor [arrow]
Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on code in PR #43947:
URL: https://github.com/apache/arrow/pull/43947#discussion_r1744701386
##########
cpp/src/parquet/encryption/encryption_internal.cc:
##########
@@ -89,6 +89,12 @@ class AesEncryptor::AesEncryptorImpl {
}
private:
+ void CheckValid() {
+ if (ctx_ == nullptr) {
Review Comment:
```suggestion
if (ARROW_PREDICT_FALSE(ctx_ == nullptr)) {
```
##########
cpp/src/parquet/encryption/encryption_internal.cc:
##########
@@ -413,6 +423,12 @@ class AesDecryptor::AesDecryptorImpl {
}
private:
+ void CheckValid() {
+ if (ctx_ == nullptr) {
Review Comment:
```suggestion
if (ARROW_PREDICT_FALSE(ctx_ == nullptr)) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org