You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@parquet.apache.org by we...@apache.org on 2019/07/18 16:28:46 UTC
[parquet-testing] branch master updated: PARQUET-1621: [C++] Add
encrypted parquet files to apache parquet-testing repository (#7)
This is an automated email from the ASF dual-hosted git repository.
wesm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/parquet-testing.git
The following commit(s) were added to refs/heads/master by this push:
new a277dc4 PARQUET-1621: [C++] Add encrypted parquet files to apache parquet-testing repository (#7)
a277dc4 is described below
commit a277dc4e55ded3e3ea27dab1e4faf98c112442df
Author: revit1976 <51...@users.noreply.github.com>
AuthorDate: Thu Jul 18 19:28:41 2019 +0300
PARQUET-1621: [C++] Add encrypted parquet files to apache parquet-testing repository (#7)
* Add encrypted parquet files to apache parquet-testing repository
* Update README.md
Add encryption keys and explanation to data/README.md
* Update README.md
Additional fixes to encrypted files section.
---
data/README.md | 33 ++++++++++++++++++++-
data/encrypt_columns_and_footer.parquet.encrypted | Bin 0 -> 4930 bytes
...ncrypt_columns_and_footer_aad.parquet.encrypted | Bin 0 -> 4938 bytes
...ncrypt_columns_and_footer_ctr.parquet.encrypted | Bin 0 -> 4864 bytes
...nd_footer_disable_aad_storage.parquet.encrypted | Bin 0 -> 4930 bytes
...rypt_columns_plaintext_footer.parquet.encrypted | Bin 0 -> 5083 bytes
data/uniform_encryption.parquet.encrypted | Bin 0 -> 5483 bytes
7 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/data/README.md b/data/README.md
index 9958630..b517afc 100644
--- a/data/README.md
+++ b/data/README.md
@@ -1,3 +1,34 @@
# Test data files for Parquet compatibility and regression testing
-TODO: Document what each file is
\ No newline at end of file
+TODO: Document what each file is
+
+## Encrypted Files
+
+Tests files with .parquet.encrypted suffix are encrypted using Parquet Modular Encryption.
+
+A detailed description of the Parquet Modular Encryption specification can be found here:
+```
+ https://github.com/apache/parquet-format/blob/encryption/Encryption.md
+```
+
+Following are the keys and key ids (when using key\_retriever) used to encrypt the encrypted columns and footer in the all the encrypted files:
+* Encrypted/Signed Footer:
+ * key: {0,1,2,3,4,5,6,7,8,9,0,1,2,3,4,5}
+ * key_id: "kf"
+* Encrypted column named double_field:
+ * key: {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,0}
+ * key_id: "kc1"
+* Encrypted column named float_field:
+ * key: {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,1}
+ * key_id: "kc2"
+
+The following files are encrypted with AAD prefix "tester":
+1. encrypt\_columns\_and\_footer\_disable\_aad\_storage.parquet.encrypted
+2. encrypt\_columns\_and\_footer\_aad.parquet.encrypted
+
+
+A sample that reads and checks these files can be found at the following tests:
+```
+cpp/src/parquet/encryption-read-configurations-test.cc
+cpp/src/parquet/test-encryption-util.h
+```
diff --git a/data/encrypt_columns_and_footer.parquet.encrypted b/data/encrypt_columns_and_footer.parquet.encrypted
new file mode 100644
index 0000000..460d05b
Binary files /dev/null and b/data/encrypt_columns_and_footer.parquet.encrypted differ
diff --git a/data/encrypt_columns_and_footer_aad.parquet.encrypted b/data/encrypt_columns_and_footer_aad.parquet.encrypted
new file mode 100644
index 0000000..863ccb2
Binary files /dev/null and b/data/encrypt_columns_and_footer_aad.parquet.encrypted differ
diff --git a/data/encrypt_columns_and_footer_ctr.parquet.encrypted b/data/encrypt_columns_and_footer_ctr.parquet.encrypted
new file mode 100644
index 0000000..0591df4
Binary files /dev/null and b/data/encrypt_columns_and_footer_ctr.parquet.encrypted differ
diff --git a/data/encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted b/data/encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted
new file mode 100644
index 0000000..abd40e7
Binary files /dev/null and b/data/encrypt_columns_and_footer_disable_aad_storage.parquet.encrypted differ
diff --git a/data/encrypt_columns_plaintext_footer.parquet.encrypted b/data/encrypt_columns_plaintext_footer.parquet.encrypted
new file mode 100644
index 0000000..0625080
Binary files /dev/null and b/data/encrypt_columns_plaintext_footer.parquet.encrypted differ
diff --git a/data/uniform_encryption.parquet.encrypted b/data/uniform_encryption.parquet.encrypted
new file mode 100644
index 0000000..048f35f
Binary files /dev/null and b/data/uniform_encryption.parquet.encrypted differ