You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@parquet.apache.org by sh...@apache.org on 2022/12/03 19:10:39 UTC
[parquet-mr] branch master updated: nested encr info (#1009)
This is an automated email from the ASF dual-hosted git repository.
shangxinli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/parquet-mr.git
The following commit(s) were added to refs/heads/master by this push:
new 433de8df3 nested encr info (#1009)
433de8df3 is described below
commit 433de8df33fcf31927f7b51456be9f53e64d48b9
Author: ggershinsky <gg...@users.noreply.github.com>
AuthorDate: Sat Dec 3 21:10:34 2022 +0200
nested encr info (#1009)
Co-authored-by: Gidon Gershinsky <gg...@apple.com>
---
parquet-hadoop/README.md | 10 +++++-----
.../main/java/org/apache/parquet/hadoop/ParquetFileWriter.java | 8 +++++++-
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/parquet-hadoop/README.md b/parquet-hadoop/README.md
index fb35954d8..5b7d2c68a 100644
--- a/parquet-hadoop/README.md
+++ b/parquet-hadoop/README.md
@@ -376,7 +376,7 @@ ParquetInputFormat to materialize records. It should be a the descendant class o
## Class: PropertiesDrivenCryptoFactory
**Property:** `parquet.encryption.column.keys`
-**Description:** List of columns to encrypt, with master key IDs (see HIVE-21848).Format: `<masterKeyID>:<colName>,<colName>;<masterKeyID>:<colName>...`
+**Description:** List of columns to encrypt, with master key IDs (see HIVE-21848).Format: `<masterKeyID>:<colName>,<colName>;<masterKeyID>:<colName>...`. Note: nested column names must be specified as full dot-separated paths for each leaf column.
**Default value:** None.
---
@@ -387,8 +387,8 @@ ParquetInputFormat to materialize records. It should be a the descendant class o
---
-**Property:** `parquet.encryption.uniform.key`
-**Description:** Master key ID for uniform encryption of all columns and footer. If set, `column.keys` and `footer.key` parameters should not be used.
+**Property:** `parquet.encryption.uniform.key`
+**Description:** Master key ID for uniform encryption of all columns and footer. If set, `column.keys` and `footer.key` parameters should not be used.
**Default value:** None.
---
@@ -456,7 +456,7 @@ If `false`, key material is stored in separate new files, created in the same fo
---
-**Property:** `parquet.encryption.kek.length.bits`
-**Description:** Length of key encryption keys (KEKs), randomly generated by parquet key management tools. Can be 128, 192 or 256 bits.
+**Property:** `parquet.encryption.kek.length.bits`
+**Description:** Length of key encryption keys (KEKs), randomly generated by parquet key management tools. Can be 128, 192 or 256 bits.
**Default value:** `128`
diff --git a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java
index a05a8898f..007eef5d1 100644
--- a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java
+++ b/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java
@@ -369,7 +369,13 @@ public class ParquetFileWriter {
for (Map.Entry<ColumnPath, ColumnEncryptionProperties> entry : columnEncryptionProperties.entrySet()) {
String[] path = entry.getKey().toArray();
if (!schema.containsPath(path)) {
- throw new ParquetCryptoRuntimeException("Encrypted column " + Arrays.toString(path) + " not in file schema");
+ StringBuilder columnList = new StringBuilder();
+ columnList.append("[");
+ for (String[] columnPath : schema.getPaths()) {
+ columnList.append(ColumnPath.get(columnPath).toDotString() + "], [");
+ }
+ throw new ParquetCryptoRuntimeException("Encrypted column [" + entry.getKey().toDotString() +
+ "] not in file schema column list: " + columnList.substring(0, columnList.length() - 3));
}
}
}