[jira] [Created] (PARQUET-2002) Parquet 1.9 not forward compatible with Parquet 1.10

["Nikhil Goyal (Jira)" <ji...@apache.org>]

Nikhil Goyal created PARQUET-2002:
-------------------------------------

             Summary: Parquet 1.9 not forward compatible with Parquet 1.10
                 Key: PARQUET-2002
                 URL: https://issues.apache.org/jira/browse/PARQUET-2002
             Project: Parquet
          Issue Type: Bug
            Reporter: Nikhil Goyal


This patch broke forward compatibility
https://issues.apache.org/jira/browse/PARQUET-357

This introduced a binary field which cannot be deserialized while reading from 1.9
New schema:
 "type": { "id": "STRING", "binary": false }
Error message
{code:java}
Caused by: java.lang.RuntimeException: shaded.parquet.org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "binary" (Class org.apache.parquet.thrift.struct.ThriftType$StringType), not marked as ignorable at [Source: java.io.StringReader@611b35d6; line: 23, column: 23] (through reference chain: org.apache.parquet.thrift.struct.StringType["binary"])
at org.apache.parquet.thrift.struct.JSON.fromJSON(JSON.java:38) at org.apache.parquet.thrift.struct.ThriftType.fromJSON(ThriftType.java:89) at org.apache.parquet.thrift.ThriftMetaData.parseDescriptor(ThriftMetaData.java:121)
... 19 more Caused by: shaded.parquet.org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "binary" (Class org.apache.parquet.thrift.struct.ThriftType$StringType), not marked as ignorable{code}
A simple fix to 1.9 should fix this

 
{code:java}
--- a/parquet-thrift/src/main/java/org/apache/parquet/thrift/struct/ThriftType.java
+++ b/parquet-thrift/src/main/java/org/apache/parquet/thrift/struct/ThriftType.java
@@ -641,11 +641,23 @@ public abstract class ThriftType {
 }
public static class StringType extends ThriftType {
+ private boolean binary = false;
+
+ @JsonIgnore
+ public boolean isBinary() {
+ return binary;
+ }
+
+ @JsonProperty
+ public void setBinary(boolean binary) {
+ this.binary = binary;
+ }
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)