You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/05/22 12:02:00 UTC
[jira] [Commented] (PARQUET-1305) Backward incompatible change
introduced in 1.8
[ https://issues.apache.org/jira/browse/PARQUET-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483835#comment-16483835 ]
ASF GitHub Bot commented on PARQUET-1305:
-----------------------------------------
nandorKollar opened a new pull request #483: PARQUET-1305: Backward incompatible change introduced in 1.8
URL: https://github.com/apache/parquet-mr/pull/483
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Backward incompatible change introduced in 1.8
> ----------------------------------------------
>
> Key: PARQUET-1305
> URL: https://issues.apache.org/jira/browse/PARQUET-1305
> Project: Parquet
> Issue Type: Bug
> Reporter: Nandor Kollar
> Assignee: Nandor Kollar
> Priority: Major
>
> ThriftSchemaConverter#convert(StructType struct) introduced a backward incompatible logic in Parquet 1.8, If StructType's [deprecated constructor|https://github.com/apache/parquet-mr/blob/master/parquet-thrift/src/main/java/org/apache/parquet/thrift/struct/ThriftType.java#L225] is used to create StructTypes, and this convert method is used, then the client receives the following exception:
> {code}
> org.apache.parquet.ShouldNeverHappenException: Encountered UNKNOWN StructOrUnionType
> at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.isUnion(ThriftSchemaConvertVisitor.java:342)
> at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:218)
> at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:74)
> at org.apache.parquet.thrift.struct.ThriftType$StructType.accept(ThriftType.java:269)
> at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.convert(ThriftSchemaConvertVisitor.java:93)
> at org.apache.parquet.thrift.ThriftSchemaConverter.convert(ThriftSchemaConverter.java:75)
> {code}
> To illustrate the issue, here's a test case, which passes on older versions, but fails on master:
> {code}
> @Test
> public void testIncompatibleThriftConverterChange() {
> ThriftSchemaConverter converter = new ThriftSchemaConverter();
> ThriftType.StructType structType = new ThriftType.StructType(
> asList(
> new ThriftField("a", (short)1, REQUIRED, new ThriftType.StringType()),
> new ThriftField("b", (short) 2, REQUIRED, new ThriftType.StringType())
> )
> );
> converter.convert(structType);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)