You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "calluw (via GitHub)" <gi...@apache.org> on 2024/01/03 16:12:04 UTC

[I] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

calluw opened a new issue, #39444:
URL: https://github.com/apache/arrow/issues/39444

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   Version: `pyarrow==14.0.2`
   
   I have been trying out the new capability introduced in Arrow 14 which allows modular encryption of Parquet files to be used alongside the generic Dataset API, but began to experience segmentation faults in C++ Arrow code when working with real, partitioned datasets rather than toy examples.
   
   The error is most often segmentation fault but I have seen all of these:
   ```
   Segmentation fault
   OSError: Failed decryption finalization
   OSError: Couldn't set AAD
   ```
   
   After some trial and error generating larger and larger toy examples until the error was reliably hit, I discovered that the threshold was when the data which is *initially written* to the dataset on disk with modular encryption was over `2^15`. In testing, `2^15` rows never faulted, but `2^15 + 1` often faults (occasionally it does not).
   
   The backtrace when triggering the fault or error always ends in the same function:
   ```
   #9  0x00007f9b726625f2 in parquet::encryption::AesDecryptor::AesDecryptorImpl::GcmDecrypt(unsigned char const*, int, unsigned char const*, int, unsigned char const*, int, unsigned char*) [clone .cold] () from venv/lib/python3.10/site-packages/pyarrow/libparquet.so.1400
   #10 0x00007f9b7270923e in parquet::Decryptor::Decrypt(unsigned char const*, int, unsigned char*) () from venv/lib/python3.10/site-packages/pyarrow/libparquet.so.1400
   #11 0x00007f9b726f569b in void parquet::ThriftDeserializer::DeserializeMessage<parquet::format::ColumnMetaData>(unsigned char const*, unsigned int*, parquet::format::ColumnMetaData*, parquet::Decryptor*) () from venv/lib/python3.10/site-packages/pyarrow/libparquet.so.1400
   #12 0x00007f9b726f8f34 in parquet::ColumnChunkMetaData::ColumnChunkMetaDataImpl::ColumnChunkMetaDataImpl(parquet::format::ColumnChunk const*, parquet::ColumnDescriptor const*, short, short, parquet::ReaderProperties const&, parquet::ApplicationVersion const*, std::shared_ptr<parquet::InternalFileDecryptor>) () from venv/lib/python3.10/site-packages/pyarrow/libparquet.so.1400
   ```
   Corresponding to this source: https://github.com/apache/arrow/blob/main/cpp/src/parquet/encryption/encryption_internal.cc#L453
   
   I found this previous fix in this area, the issue for which details the exact same symptoms: https://github.com/apache/arrow/commit/88bccab18a4ab818355209e45862cc52f9cf4a0d
   ```
   However, AesDecryptor::AesDecryptorImpl::GcmDecrypt() and AesDecryptor::AesDecryptorImpl::CtrDecrypt() use ctx_ member of type EVP_CIPHER_CTX from OpenSSL, which shouldn't be used from multiple threads concurrently.
   ```
   So I was suspicious that it could be a multi-threading issue during read or write.
   
   Whilst attempting to narrow down the cause (and whether the root cause occurs during writing or reading), I made the following observations:
   - Writing `2^15 + 1` rows, deleting half the dataset and then reading in the full dataset still encounters the error
   - Writing `2^15 + 1` rows, and filtering to half the partitions in the dataset when reading still encounters the error
   - Writing `2^15 + 1` rows, and decrypting with modular encryption each individual parquet file in the dataset and concatenating them never failed, only doing so as a full dataset
   - Writing or reading or both with `use_threads=False` still encounters the error
   
   The fact that the issue still occurs when the dataset on disk is halved and then read again suggested corruption during write, but the fact that all individual Parquet files are still independently readable suggests an issue with modular decryption during Dataset operations. Issue still occurring with threading disabled was unexpected.
   
   The error is reproducible using any random data, and using a KMS client which actually does no encryption at all (eliminating our custom KMS client as a probable cause), but simply passes the keys used around in (encoded) plaintext. It occurs whether the footer is plaintext or not, or whether envelope encryption is used or not.
   
   Reproduction in Python here (no partitions needed at all, so it produces a single Parquet file, which can be read normally):
   ```python3
   import base64
   import numpy as np
   import tempfile
   
   import pyarrow as pa
   import pyarrow.parquet as pq
   import pyarrow.dataset as ds
   import pyarrow.parquet.encryption as pqe
   
   class NoOpKmsClient(pqe.KmsClient):
       def __init__(self):
           super().__init__()
   
       def wrap_key(self, key_bytes: bytes, _: str) -> bytes:
           b = base64.b64encode(key_bytes)
           return b
   
       def unwrap_key(self, wrapped_key: bytes, _: str) -> bytes:
           b = base64.b64decode(wrapped_key)
           return b
   
   row_count = pow(2, 15) + 1
   table = pa.Table.from_arrays([pa.array(np.random.rand(row_count), type=pa.float32())], names=["foo"])
   
   kms_config = pqe.KmsConnectionConfig()
   crypto_factory = pqe.CryptoFactory(lambda _: NoOpKmsClient())
   encryption_config = pqe.EncryptionConfiguration(
       footer_key="UNIMPORTANT_KEY",
       column_keys={"UNIMPORTANT_KEY": ["foo"]},
       double_wrapping=True,
       plaintext_footer=False,
       data_key_length_bits=128,
   )
   pqe_config = ds.ParquetEncryptionConfig(crypto_factory, kms_config, encryption_config)
   pqd_config = ds.ParquetDecryptionConfig(crypto_factory, kms_config, pqe.DecryptionConfiguration())
   scan_options = ds.ParquetFragmentScanOptions(decryption_config=pqd_config)
   file_format = ds.ParquetFileFormat(default_fragment_scan_options=scan_options)
   write_options = file_format.make_write_options(encryption_config=pqe_config)
   file_decryption_properties = crypto_factory.file_decryption_properties(kms_config)
   
   with tempfile.TemporaryDirectory() as tempdir:
       path = tempdir + "/test-dataset"
       ds.write_dataset(table, path, format=file_format, file_options=write_options)
   
       file_path = path + "/part-0.parquet"
       new_table = pq.ParquetFile(file_path, decryption_properties=file_decryption_properties).read()
       assert table == new_table
   
       dataset = ds.dataset(path, format=file_format)
       new_table = dataset.to_table()
       assert table == new_table
   ```
   
   Any help here would be much appreciated: being restricted to `2^15` rows is a roadblock for us for the use of this feature.
   
   ### Component(s)
   
   Parquet, Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "adamreeve (via GitHub)" <gi...@apache.org>.
adamreeve commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1891107716

   I believe that the reason this issue is only seen with more than 2^15 rows is that this is the value of `kMaxBatchSize` used in Acero, and when writing the dataset, if there are more rows than this, the Parquet file is split into multiple row groups. As a workaround, I can get the reproduction code from the issue description to work without error if I use `ds.write_dataset(table, path, format=file_format, file_options=write_options, min_rows_per_group=row_count)`, so that Parquet files only ever have one row group.
   
   The reason this leads to crashes when multi-threading is enabled seems to be that when scanning the dataset, `RowGroupGenerator::read_one_grow_group` is called concurrently from different threads due to using a `ReadaheadGenerator`, leading to concurrent use of the same `AesDecryptor` instances, which are not thread safe. Just putting mutexes around use of the `AesDecryptor`s isn't sufficient to fix the problem due to `Decryptor::UpdateAad` being used to update the `AAD` value as data pages are read, which then affects the use of the decryptor from other threads. The `InternalFileDecryptor::Get*Decryptor` methods are also called concurrently but are not thread safe due to modifying `std::map`s and the `all_decryptors_` vector.
   
   @eirki mentioned seeing this issue without using the Dataset API, and I believe this might also happen when `FileReaderImpl::DecodeRowGroups` is used to decode multiple row groups concurrently when threading is enabled, but I haven't tried reproducing that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "AudriusButkevicius (via GitHub)" <gi...@apache.org>.
AudriusButkevicius commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-2004791873

   Any chances of a patch release with this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1883125197

   > > If I disable ARROW_ENABLE_THREADING on the C++ side on a Apple M1 Max, I do not hit the exception.
   > 
   > Do you mean it can be stably reproduced when ARROW_ENABLE_THREADING is ON? @tolleybot
   
   It occurs intermittently when ARROW_ENABLE_THREADING is enabled. On my Apple M1 Max, the exception doesn't happen every single time I run the test, but it does occur quite frequently.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "eirki (via GitHub)" <gi...@apache.org>.
eirki commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1887256024

   Just to add to the conversation: I have been have the same problem (at least I think it is the same problem) without using the Dataset API, and with tables with fewer than 2^15 rows (as little as 30 500). This occured with non-script-generated data, and I have been having all manner of trouble recreating it. It seems to be occuring in some files more often than others.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1885044357

   Hi @tolleybot , I am currently working on the 15.0.0 release. If someone is able to push a fix before I create a RC in the next days I am happy to cherry-pick the fix but I don't think we should hold the release, otherwise would be difficult to release due to other bugs/issues. If the issue is solved after the release is done we can also add it to a possible patch release and or to the scheduled quarter release. Of course if there are major concerns and other community members think this should block the release I will act accordingly.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "jorisvandenbossche (via GitHub)" <gi...@apache.org>.
jorisvandenbossche commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1885051927

   (also, as far as I understand, while it is definitely a serious bug, it's already present in the previous major release as well, so it's not a regression (or at least not a new regression), which I think makes it a little bit less blocking)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1892783187

   I have submitted a pull request that addresses this issue: [#39623](https://github.com/apache/arrow/pull/39623). This PR contains the proposed fix for the segmentation fault encountered in modular encryption as discussed here.
   @wgtmac 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1889812752

   **I changed GetColumnDecryptor so that it always returns a new Decryptor and I no longer see the issue.  I am not sure , without testing, if I am stomping on something else, or if this is efficient since the Decryptor was being cached. **
   
   ### Previous Method:
   
   ```
   std::shared_ptr<Decryptor> InternalFileDecryptor::GetColumnDecryptor(
       const std::string& column_path, const std::string& column_key_metadata,
       const std::string& aad, bool metadata) {
     std::string column_key;
     // first look if we already got the decryptor from before
     if (metadata) {
       if (column_metadata_map_.find(column_path) != column_metadata_map_.end()) {
         auto res(column_metadata_map_.at(column_path));
         res->UpdateAad(aad);
         return res;
       }
     } else {
       if (column_data_map_.find(column_path) != column_data_map_.end()) {
         auto res(column_data_map_.at(column_path));
         res->UpdateAad(aad);
         return res;
       }
     }
   
     column_key = properties_->column_key(column_path);
     // No explicit column key given via API. Retrieve via key metadata.
     if (column_key.empty() && !column_key_metadata.empty() &&
         properties_->key_retriever() != nullptr) {
       try {
         column_key = properties_->key_retriever()->GetKey(column_key_metadata);
       } catch (KeyAccessDeniedException& e) {
         std::stringstream ss;
         ss << "HiddenColumnException, path=" + column_path + " " << e.what() << "\n";
         throw HiddenColumnException(ss.str());
       }
     }
     if (column_key.empty()) {
       throw HiddenColumnException("HiddenColumnException, path=" + column_path);
     }
   
     // Create both data and metadata decryptors to avoid redundant retrieval of key
     // using the key_retriever.
     int key_len = static_cast<int>(column_key.size());
     auto aes_metadata_decryptor = encryption::AesDecryptor::Make(
         algorithm_, key_len, /*metadata=*/true, &all_decryptors_);
     auto aes_data_decryptor = encryption::AesDecryptor::Make(
         algorithm_, key_len, /*metadata=*/false, &all_decryptors_);
   
     column_metadata_map_[column_path] = std::make_shared<Decryptor>(
         aes_metadata_decryptor, column_key, file_aad_, aad, pool_);
     column_data_map_[column_path] =
         std::make_shared<Decryptor>(aes_data_decryptor, column_key, file_aad_, aad, pool_);
   
     if (metadata) return column_metadata_map_[column_path];
     return column_data_map_[column_path];
   }
   ```
   
   ### Updated Method
   
   ```
   std::shared_ptr<Decryptor> InternalFileDecryptor::GetColumnDecryptor(
       const std::string& column_path, const std::string& column_key_metadata,
       const std::string& aad, bool metadata) {
     std::string column_key = properties_->column_key(column_path);
   
     column_key = properties_->column_key(column_path);
     // No explicit column key given via API. Retrieve via key metadata.
     if (column_key.empty() && !column_key_metadata.empty() &&
         properties_->key_retriever() != nullptr) {
       try {
         column_key = properties_->key_retriever()->GetKey(column_key_metadata);
       } catch (KeyAccessDeniedException& e) {
         std::stringstream ss;
         ss << "HiddenColumnException, path=" + column_path + " " << e.what() << "\n";
         throw HiddenColumnException(ss.str());
       }
     }
     if (column_key.empty()) {
       throw HiddenColumnException("HiddenColumnException, path=" + column_path);
     }
   
     int key_len = static_cast<int>(column_key.size());
     auto aes_decryptor =
         encryption::AesDecryptor::Make(algorithm_, key_len, metadata, &all_decryptors_);
   
     return std::make_shared<Decryptor>(aes_decryptor, column_key, file_aad_, aad, pool_);
   ```
   }
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1883820555

   The error is occurring in the decryption of the file.  As a test, I wrote the file using a single thread to be safe.  I then decrypt the file multiple times with multi-threading enabled.  I again got some failures and some tests which did not fail.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #39444: [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows
URL: https://github.com/apache/arrow/issues/39444


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "calluw (via GitHub)" <gi...@apache.org>.
calluw commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1888757088

   I can confirm that reducing the batch size via PyArrow (`dataset.to_table(batch_size=1000)` in my reproduction above) does prevent the error, with a dataset of size `2^15 + 1` rows. In fact, specifically any batch size up to and including 2048 (`2^11`) works, but `2^11 + 1` hits the error! This was true also for `2^16 + 1` rows, however at `2^17 + 1` rows and above, no batch size was sufficiently small to not trigger the error.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "calluw (via GitHub)" <gi...@apache.org>.
calluw commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1891826511

   Yes, that does explain why the max batch size that I could get to work was `2^11` (`16 == 2^4` -> `2^4 * 2^11 == 2^15`). It also explains why this didn't vary with increased row size, since `2^15` is the turning point at which multiple row groups are written. Still not clear why `2^17` and above couldn't be made to work with batch size `2^11` since it should matter, but I can confirm that setting `batch_readahead=0` does work. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1875652274

   cc @wgtmac 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Python][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1881219059

   @wgtmac @mapleFU @jorisvandenbossche should this be a blocker for the release?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tmct (via GitHub)" <gi...@apache.org>.
tmct commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1884512804

   @tolleybot very interesting, thanks. Is that still through pyarrow still, but with a no-threading cpp build?
   
   What is very strange is how @calluw and I still hit this reliably with use_threads=False (write + read) then!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1885027895

   I concur with @raulcd concern regarding this issue as a blocker for Apache Arrow release 15.0.0.  I would like to suggest to keep this as a milestone for 15.
   This issue is critical due to its direct impact on stability and reliability, especially since it's reproducible, indicating a systemic problem. The involvement of encrypted datasets raises concerns over data integrity and security. Prioritizing this fix will uphold the quality expected in a major release.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1888017682

   I created this simple C++ program that recreates the issue.  I found that if I hardcoded the batch size as follows the problem never occurs.  Optionally if you set the number of threads to one the problem also is gone.
   Hopefully this will provide some ideas!
   
   ```
     if (!scanner_builder->BatchSize(1000).ok()) {
       throw std::runtime_error("Failed to set batch size.");
     }
     
   ```
    **config.txt used to set values dynamically is just a text file with tow rows of:**
    62780
   4
   
   
   ### **Sample code:**
   
   ```
   #include <arrow/api.h>
   #include <arrow/dataset/dataset.h>
   #include <arrow/dataset/file_base.h>
   #include <arrow/dataset/file_parquet.h>
   #include <arrow/dataset/parquet_encryption_config.h>
   #include <arrow/filesystem/localfs.h>
   #include <arrow/testing/random.h>
   #include <arrow/util/base64.h>
   #include <parquet/arrow/reader.h>
   #include <parquet/encryption/crypto_factory.h>
   #include <parquet/encryption/encryption.h>
   #include <parquet/encryption/encryption_internal.h>
   #include <parquet/encryption/kms_client.h>
   #include <cmath>
   #include <cstdlib>
   #include <filesystem>
   #include <fstream>
   #include <iostream>
   #include <limits>
   #include <memory>
   #include <string_view>
   
   using namespace arrow;
   using namespace arrow::dataset;
   using namespace parquet::encryption;
   
   constexpr std::string_view kFooterKeyName = "footer_key";
   constexpr std::string_view kColumnKeyMapping = "col_key: foo";
   constexpr std::string_view kBaseDir = "/Users/dtolley/Documents/temp";
   
   class NoOpKmsClient : public parquet::encryption::KmsClient {
    public:
     NoOpKmsClient() {}
   
     std::string WrapKey(const std::string& key_bytes,
                         const std::string& master_key_identifier) override {
       // Base64 encode the key
       return arrow::util::base64_encode(std::string_view(key_bytes));
     }
   
     std::string UnwrapKey(const std::string& wrapped_key,
                           const std::string& master_key_identifier) override {
       // Base64 decode the key
       return arrow::util::base64_decode(std::string_view(wrapped_key));
     }
   };
   
   class NoOpKmsClientFactory : public parquet::encryption::KmsClientFactory {
    public:
     NoOpKmsClientFactory() {}
   
     std::shared_ptr<parquet::encryption::KmsClient> CreateKmsClient(
         const parquet::encryption::KmsConnectionConfig& config) override {
       // Return a new instance of NoOpKmsClient
       return std::make_shared<NoOpKmsClient>();
     }
   };
   
   void CreateRandomTable(std::shared_ptr<arrow::Table>& table, int64_t row_count) {
     // Using a FloatBuilder to create the array
     arrow::FloatBuilder float_builder;
     arrow::Status status;
   
     // Append values to the builder
    for (int64_t i = 0; i < row_count; ++i) {
       float value = static_cast<float>(i); 
       status = float_builder.Append(value);
       if (!status.ok()) {
         throw std::runtime_error("Error appending value to FloatBuilder: " +
                                  status.ToString());
       }
     }
   
     // Finalize the builder to get the array
     std::shared_ptr<arrow::Array> foo_array;
     status = float_builder.Finish(&foo_array);
     if (!status.ok()) {
       throw std::runtime_error("Error finalizing FloatBuilder: " + status.ToString());
     }
   
     // Create the table
     auto foo_field = arrow::field("foo", arrow::float32());
     auto schema = arrow::schema({foo_field});
     table = arrow::Table::Make(schema, {foo_array});
   }
   
   int main() {
     std::ifstream file("config.txt");  
   
     if (!file) {
       std::cerr << "Unable to open file config.txt";
       return 1;  // call system to stop
     }
   
     int ROW_COUNT;
     file >> ROW_COUNT;
     int NUM_THREADS;
     file >> NUM_THREADS;
     file.close();
   
     std::string num_threads_str = std::to_string(NUM_THREADS);
     setenv("OMP_NUM_THREADS", num_threads_str.c_str(), 1);
   
     std::cout << "Max value of short int: " << std::numeric_limits<short int>::max()
               << std::endl;
     std::cout << "ROW_COUNT: " << ROW_COUNT << std::endl;
     std::cout << "NUM_THREADS: " << NUM_THREADS << std::endl;
     try {
       auto file_system = std::make_shared<fs::LocalFileSystem>();
   
       std::shared_ptr<Table> table;
       CreateRandomTable(table, ROW_COUNT);
   
       auto crypto_factory = std::make_shared<CryptoFactory>();
       auto kms_client_factory = std::make_shared<NoOpKmsClientFactory>();
       crypto_factory->RegisterKmsClientFactory(std::move(kms_client_factory));
       auto kms_connection_config = std::make_shared<KmsConnectionConfig>();
   
       auto encryption_config =
           std::make_shared<EncryptionConfiguration>(std::string(kFooterKeyName));
       encryption_config->column_keys = kColumnKeyMapping;
       auto parquet_encryption_config = std::make_shared<ParquetEncryptionConfig>();
       parquet_encryption_config->crypto_factory = crypto_factory;
       parquet_encryption_config->kms_connection_config = kms_connection_config;
       parquet_encryption_config->encryption_config = std::move(encryption_config);
   
   #if 1
       // Writing dataset
       {
         // cleanup
         for (const auto& entry : std::filesystem::directory_iterator(kBaseDir)) {
           if (entry.path().extension() == ".parquet") {
             std::filesystem::remove(entry.path());
           }
         }
   
         auto file_format = std::make_shared<ParquetFileFormat>();
   
         auto parquet_file_write_options =
             arrow::internal::checked_pointer_cast<ParquetFileWriteOptions>(
                 file_format->DefaultWriteOptions());
         parquet_file_write_options->parquet_encryption_config = parquet_encryption_config;
   
         auto dataset = std::make_shared<InMemoryDataset>(table);
         auto scanner_builder_result = dataset->NewScan();
         if (!scanner_builder_result.ok()) {
           throw std::runtime_error("Failed to create scanner builder.");
         }
         auto scanner_builder = *scanner_builder_result;
         auto scanner_result = scanner_builder->Finish();
         if (!scanner_result.ok()) {
           throw std::runtime_error("Failed to create scanner.");
         }
         auto scanner = *scanner_result;
   
         FileSystemDatasetWriteOptions write_options;
         write_options.file_write_options = parquet_file_write_options;
         write_options.filesystem = file_system;
         write_options.base_dir = kBaseDir;
         write_options.basename_template = "part{i}.parquet";
         write_options.partitioning = std::make_shared<DirectoryPartitioning>(schema({}));
         auto write_result = FileSystemDataset::Write(write_options, std::move(scanner));
         if (!write_result.ok()) {
           throw std::runtime_error("Failed to write dataset.");
         }
       }
   #endif
   
       // Reading dataset
       {
         auto decryption_config = std::make_shared<DecryptionConfiguration>();
         auto parquet_decryption_config = std::make_shared<ParquetDecryptionConfig>();
         parquet_decryption_config->crypto_factory = crypto_factory;
         parquet_decryption_config->kms_connection_config = kms_connection_config;
         parquet_decryption_config->decryption_config = std::move(decryption_config);
   
         auto file_format = std::make_shared<ParquetFileFormat>();
         auto parquet_scan_options = std::make_shared<ParquetFragmentScanOptions>();
         parquet_scan_options->parquet_decryption_config = parquet_decryption_config;
         file_format->default_fragment_scan_options = std::move(parquet_scan_options);
   
         fs::FileSelector selector;
         selector.base_dir = kBaseDir;
         selector.recursive = true;
   
         FileSystemFactoryOptions factory_options;
         factory_options.partition_base_dir = kBaseDir;
         auto dataset_factory_result = FileSystemDatasetFactory::Make(
             file_system, selector, file_format, factory_options);
         if (!dataset_factory_result.ok()) {
           throw std::runtime_error("Failed to create dataset factory.");
         }
         auto dataset_factory = *dataset_factory_result;
   
         auto dataset_result = dataset_factory->Finish();
         if (!dataset_result.ok()) {
           throw std::runtime_error("Failed to finish dataset factory.");
         }
         auto dataset = *dataset_result;
   
         auto scanner_builder_result = dataset->NewScan();
         if (!scanner_builder_result.ok()) {
           throw std::runtime_error("Failed to create scanner builder.");
         }
         auto scanner_builder = *scanner_builder_result;
         // **** REMOVE THE FOLLOWING FOR DECRYPTION TO FAIL****//
         if (!scanner_builder->BatchSize(1000).ok()) {
           throw std::runtime_error("Failed to set batch size.");
         }
   
         if (!scanner_builder->UseThreads(false).ok()) {
           throw std::runtime_error(
               "Failed to set scanner builder to single-threaded mode.");
         }
   
         auto scanner_result = scanner_builder->Finish();
         if (!scanner_result.ok()) {
           throw std::runtime_error("Failed to finish scanner.");
         }
         auto scanner = *scanner_result;
   
         auto read_table_result = scanner->ToTable();
         if (!read_table_result.ok()) {
           throw std::runtime_error("Failed to read table from scanner.");
         }
         auto read_table = *read_table_result;
       }
   
       std::cout << "Dataset written and read successfully." << std::endl;
     } catch (const std::exception& e) {
       std::cerr << "Error: " << e.what() << std::endl;
       return 1;
     }
   
     return 0;
   }
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Python][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1881688413

   If I disable ARROW_ENABLE_THREADING on the C++ side on a Apple M1 Max, I do not hit the exception.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Python][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1882284962

   > ARROW_ENABLE_THREADING
   
   Do you mean it can be stably reproduced when ARROW_ENABLE_THREADING is ON?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1884996689

   @tmct I recompile the C++ build using  ARROW_ENABLE_THREADING = OFF and I have no issue.  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #39444: [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows
URL: https://github.com/apache/arrow/issues/39444


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "wgtmac (via GitHub)" <gi...@apache.org>.
wgtmac commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1890495277

   @tolleybot Thanks for the detail code! Is it possible to convert the piece of code that you can stably reproduce the issue into an unit test? And then open an PR together with your proposed fix? I'd happy to check it out on my end.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [C++][Parquet] Segmentation fault reading modular encrypted Parquet dataset over 2^15 rows [arrow]

Posted by "tolleybot (via GitHub)" <gi...@apache.org>.
tolleybot commented on issue #39444:
URL: https://github.com/apache/arrow/issues/39444#issuecomment-1890510214

   Yes, I can. 👌 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org