[jira] [Commented] (PARQUET-405) Backwards-incompatible change to thrift metadata

["Ben Kirwin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PARQUET-405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056802#comment-15056802 ] 

Ben Kirwin commented on PARQUET-405:
------------------------------------

Failure stacktrace:

{code}
Caused by: org.apache.parquet.ShouldNeverHappenException: Encountered UNKNOWN StructOrUnionType
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.isUnion(ThriftSchemaConvertVisitor.java:337)
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:213)
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:76)
    at org.apache.parquet.thrift.struct.ThriftType$StructType.accept(ThriftType.java:272)
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.convert(ThriftSchemaConvertVisitor.java:88)
    at org.apache.parquet.thrift.ThriftSchemaConverter.convert(ThriftSchemaConverter.java:60)
    at org.apache.parquet.thrift.ThriftRecordConverter.<init>(ThriftRecordConverter.java:797)
    at org.apache.parquet.scrooge.ScroogeRecordConverter.<init>(ScroogeRecordConverter.java:36)
{code}

> Backwards-incompatible change to thrift metadata
> ------------------------------------------------
>
>                 Key: PARQUET-405
>                 URL: https://issues.apache.org/jira/browse/PARQUET-405
>             Project: Parquet
>          Issue Type: Bug
>    Affects Versions: 1.8.0
>            Reporter: Ben Kirwin
>
> Sometime in the last few versions, a {{isStructOrUnion}} field has been added to the `thrift.descriptor` written to the parquet header:
> {code}
> {
>     "children": [ ... ],
>     "id": "STRUCT", 
>     "structOrUnionType": "STRUCT"
> }
> {code}
> The current release now throws an exception when that field is missing  / {{UNKNOWN}}). This makes it impossible to read back thrift data written using a previous release.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)