Re: [PR] GH-41719: [C++][Parquet] Cannot read encrypted parquet datasets via _metadata file [arrow]

["rok (via GitHub)" <gi...@apache.org>]

rok commented on code in PR #41821:
URL: https://github.com/apache/arrow/pull/41821#discussion_r1706158737


##########
cpp/src/parquet/file_writer.cc:
##########
@@ -567,6 +567,44 @@ void WriteEncryptedFileMetadata(const FileMetaData& file_metadata,
   }
 }
 
+void WriteEncryptedMetadataFile(
+    const FileMetaData& metadata, std::shared_ptr<::arrow::io::OutputStream> sink,
+    std::shared_ptr<FileEncryptionProperties> file_encryption_properties) {
+  auto file_encryptor = std::make_unique<InternalFileEncryptor>(
+      file_encryption_properties.get(), ::arrow::default_memory_pool());
+
+  if (file_encryption_properties->encrypted_footer()) {
+    PARQUET_THROW_NOT_OK(sink->Write(kParquetEMagic, 4));
+
+    PARQUET_ASSIGN_OR_THROW(int64_t position, sink->Tell());
+    auto metadata_start = static_cast<uint64_t>(position);
+
+    auto writer_props = parquet::WriterProperties::Builder()
+                            .encryption(file_encryption_properties)
+                            ->build();
+    auto builder = FileMetaDataBuilder::Make(metadata.schema(), writer_props);
+
+    auto footer_metadata = builder->Finish(metadata.key_value_metadata());
+    auto crypto_metadata = builder->GetCryptoMetaData();
+    WriteFileCryptoMetaData(*crypto_metadata, sink.get());
+
+    auto footer_encryptor = file_encryptor->GetFooterEncryptor();
+    WriteEncryptedFileMetadata(metadata, sink.get(), footer_encryptor, true);
+    PARQUET_ASSIGN_OR_THROW(position, sink->Tell());
+    auto footer_and_crypto_len = static_cast<uint32_t>(position - metadata_start);
+    PARQUET_THROW_NOT_OK(
+        sink->Write(reinterpret_cast<uint8_t*>(&footer_and_crypto_len), 4));
+    PARQUET_THROW_NOT_OK(sink->Write(kParquetEMagic, 4));
+  } else {

Review Comment:
   I'm doing metadata encryption similarly to approach here: https://github.com/apache/arrow/blob/a4d58e0c8213cd14d0953c75eb047ea1693fddc9/cpp/src/parquet/file_writer.cc#L416-L432
   However it seems decryption fails (see below) when using `RowGroup` metadata (after deserialization and decryption).
   https://github.com/apache/arrow/blob/4d5041a3eff486cd82c6790f6dae63fba2321474/cpp/src/arrow/dataset/file_parquet.cc#L1084-L1086 This makes me think I'm either not serializing correctly or there's an issue with encryption/decryption properties I'm supplying.
   
   @wgtmac @pitrou does anything obviously wrong stands out here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org