You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/01/12 14:11:00 UTC
[jira] [Commented] (PARQUET-2103) crypto exception in print toPrettyJSON
[ https://issues.apache.org/jira/browse/PARQUET-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676041#comment-17676041 ]
ASF GitHub Bot commented on PARQUET-2103:
-----------------------------------------
wgtmac commented on code in PR #1019:
URL: https://github.com/apache/parquet-mr/pull/1019#discussion_r1068164918
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/metadata/FileMetaData.java:
##########
@@ -71,7 +79,7 @@ public MessageType getSchema() {
@Override
public String toString() {
- return "FileMetaData{schema: "+schema+ ", metadata: " + keyValueMetaData + "}";
+ return "FileMetaData{schema: "+schema+ ", metadata: " + keyValueMetaData + ", encryption: " + encryptionType + "}";
Review Comment:
```suggestion
return "FileMetaData{schema: " + schema + ", metadata: " + keyValueMetaData + ", encryption: " + encryptionType + "}";
```
##########
parquet-hadoop/src/main/java/org/apache/parquet/hadoop/metadata/FileMetaData.java:
##########
@@ -50,16 +51,23 @@ public final class FileMetaData implements Serializable {
* @throws NullPointerException if schema or keyValueMetaData is {@code null}
*/
public FileMetaData(MessageType schema, Map<String, String> keyValueMetaData, String createdBy) {
- this(schema, keyValueMetaData, createdBy, null);
+ this(schema, keyValueMetaData, createdBy, EncryptionType.UNENCRYPTED, null);
}
-
- public FileMetaData(MessageType schema, Map<String, String> keyValueMetaData, String createdBy, InternalFileDecryptor fileDecryptor) {
+
+ public FileMetaData(MessageType schema, Map<String, String> keyValueMetaData, String createdBy,
+ InternalFileDecryptor fileDecryptor) {
+ this(schema, keyValueMetaData, createdBy, null, fileDecryptor);
Review Comment:
```suggestion
this(schema, keyValueMetaData, createdBy, EncryptionType.UNENCRYPTED, fileDecryptor);
```
> crypto exception in print toPrettyJSON
> --------------------------------------
>
> Key: PARQUET-2103
> URL: https://issues.apache.org/jira/browse/PARQUET-2103
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: 1.12.0, 1.12.1, 1.12.2, 1.12.3
> Reporter: Gidon Gershinsky
> Assignee: Gidon Gershinsky
> Priority: Minor
>
> In debug mode, this code
> {{if (LOG.isDebugEnabled()) {}}
> {{ LOG.debug(ParquetMetadata.toPrettyJSON(parquetMetadata));}}
> {{}}}
> called in {{org.apache.parquet.format.converter.ParquetMetadataConverter.readParquetMetadata()}}
>
> _*in encrypted files with plaintext footer*_
> triggers an exception:
>
> {{Caused by: org.apache.parquet.crypto.ParquetCryptoRuntimeException: [id]. Null File Decryptor }}
> {{ at org.apache.parquet.hadoop.metadata.EncryptedColumnChunkMetaData.decryptIfNeeded(ColumnChunkMetaData.java:602) ~[parquet-hadoop-1.12.0jar:1.12.0]}}
> {{ at org.apache.parquet.hadoop.metadata.ColumnChunkMetaData.getEncodingStats(ColumnChunkMetaData.java:353) ~[parquet-hadoop-1.12.0jar:1.12.0]}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]}}
> {{ at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]}}
> {{ at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]}}
> {{ at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:689) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:755) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1516) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ObjectWriter._writeValueAndClose(ObjectWriter.java:1217) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at shaded.parquet.com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1059) ~[parquet-jackson-1.12.0jar:1.12.0]}}
> {{ at org.apache.parquet.hadoop.metadata.ParquetMetadata.toJSON(ParquetMetadata.java:68) ~[parquet-hadoop-1.12.0jar:1.12.0]}}
> {{ ... 23 more}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)