[jira] [Updated] (HIVE-22178) Parquet FilterPredicate throws CastException after SchemaEvolution.

["Naresh P R (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-22178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Naresh P R updated HIVE-22178:
------------------------------
    Attachment: HIVE-22178.1.patch
        Status: Patch Available  (was: In Progress)

> Parquet FilterPredicate throws CastException after SchemaEvolution.
> -------------------------------------------------------------------
>
>                 Key: HIVE-22178
>                 URL: https://issues.apache.org/jira/browse/HIVE-22178
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.1.0
>            Reporter: Naresh P R
>            Assignee: Naresh P R
>            Priority: Major
>         Attachments: HIVE-22178.1.patch
>
>
> Below are the repro steps.
> {code:java}
> create table parq_test(age int, name string) stored as parquet;
> insert into parq_test values(1, 'aaaa');
> alter table parq_test change age age string;
> insert into parq_test values('b', 'bbbb');
> select * from parq_test where age='b';{code}
> Exception thrown after changing column datatype is below
> {code:java}
> Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's declared type (org.apache.parquet.io.api.Binary) does not match the schema found in file metadata. Column age is of type: INT32
> Valid types for this column are: [class java.lang.Integer]
>  at org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
>  at org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
>  at org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
>  at org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
>  at org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
>  at org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
>  at org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
>  at org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
>  at org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
>  at org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
>  at org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
>  at org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
>  at org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.<init>(VectorizedParquetRecordReader.java:147)
>  ... 31 more{code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)