You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by GitBox <gi...@apache.org> on 2022/05/18 11:26:48 UTC
[GitHub] [parquet-mr] ggershinsky opened a new pull request, #969: PARQUET-2148: Enable uniform decryption with plaintext footer
ggershinsky opened a new pull request, #969:
URL: https://github.com/apache/parquet-mr/pull/969
Currently, uniform decryption is not enabled in the plaintext footer mode - for no good reason. Column metadata is available, we just need to decrypt and use it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli commented on a diff in pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer
Posted by GitBox <gi...@apache.org>.
shangxinli commented on code in PR #969:
URL: https://github.com/apache/parquet-mr/pull/969#discussion_r876012014
##########
parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java:
##########
@@ -1556,25 +1558,32 @@ public ParquetMetadata fromParquetMetadata(FileMetaData parquetMetadata,
} else { // Encrypted column
boolean encryptedWithFooterKey = cryptoMetaData.isSetENCRYPTION_WITH_FOOTER_KEY();
if (encryptedWithFooterKey) { // Column encrypted with footer key
- if (!encryptedFooter) {
- throw new ParquetCryptoRuntimeException("Column encrypted with footer key in file with plaintext footer");
+ if (null == fileDecryptor) {
+ throw new ParquetCryptoRuntimeException("Column encrypted with footer key: No keys available");
}
if (null == metaData) {
throw new ParquetCryptoRuntimeException("ColumnMetaData not set in Encryption with Footer key");
}
- if (null == fileDecryptor) {
- throw new ParquetCryptoRuntimeException("Column encrypted with footer key: No keys available");
- }
columnPath = getPath(metaData);
+ if (!encryptedFooter) { // Unencrypted footer. Decrypt full column metadata, using footer key
Review Comment:
I am not following why we use the footer key to decrypt? Wasn't the column key used before?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] ggershinsky commented on a diff in pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer
Posted by GitBox <gi...@apache.org>.
ggershinsky commented on code in PR #969:
URL: https://github.com/apache/parquet-mr/pull/969#discussion_r876022965
##########
parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java:
##########
@@ -1556,25 +1558,32 @@ public ParquetMetadata fromParquetMetadata(FileMetaData parquetMetadata,
} else { // Encrypted column
boolean encryptedWithFooterKey = cryptoMetaData.isSetENCRYPTION_WITH_FOOTER_KEY();
if (encryptedWithFooterKey) { // Column encrypted with footer key
- if (!encryptedFooter) {
- throw new ParquetCryptoRuntimeException("Column encrypted with footer key in file with plaintext footer");
+ if (null == fileDecryptor) {
+ throw new ParquetCryptoRuntimeException("Column encrypted with footer key: No keys available");
}
if (null == metaData) {
throw new ParquetCryptoRuntimeException("ColumnMetaData not set in Encryption with Footer key");
}
- if (null == fileDecryptor) {
- throw new ParquetCryptoRuntimeException("Column encrypted with footer key: No keys available");
- }
columnPath = getPath(metaData);
+ if (!encryptedFooter) { // Unencrypted footer. Decrypt full column metadata, using footer key
Review Comment:
with uniform encryption, we leverage the "encrypt_with_footer_key" mode in parquet-format
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] ggershinsky commented on a diff in pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer
Posted by GitBox <gi...@apache.org>.
ggershinsky commented on code in PR #969:
URL: https://github.com/apache/parquet-mr/pull/969#discussion_r876028764
##########
parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java:
##########
@@ -1556,25 +1558,32 @@ public ParquetMetadata fromParquetMetadata(FileMetaData parquetMetadata,
} else { // Encrypted column
boolean encryptedWithFooterKey = cryptoMetaData.isSetENCRYPTION_WITH_FOOTER_KEY();
if (encryptedWithFooterKey) { // Column encrypted with footer key
- if (!encryptedFooter) {
- throw new ParquetCryptoRuntimeException("Column encrypted with footer key in file with plaintext footer");
+ if (null == fileDecryptor) {
+ throw new ParquetCryptoRuntimeException("Column encrypted with footer key: No keys available");
}
if (null == metaData) {
throw new ParquetCryptoRuntimeException("ColumnMetaData not set in Encryption with Footer key");
}
- if (null == fileDecryptor) {
- throw new ParquetCryptoRuntimeException("Column encrypted with footer key: No keys available");
- }
columnPath = getPath(metaData);
+ if (!encryptedFooter) { // Unencrypted footer. Decrypt full column metadata, using footer key
Review Comment:
this already works ok with encrypted footers; now fixing for plaintext footers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [parquet-mr] shangxinli merged pull request #969: PARQUET-2148: Enable uniform decryption with plaintext footer
Posted by GitBox <gi...@apache.org>.
shangxinli merged PR #969:
URL: https://github.com/apache/parquet-mr/pull/969
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org