[jira] [Created] (HIVE-11031) ORC concatenation of old files can fail while merging column statistics

["Prasanth Jayachandran (JIRA)" <ji...@apache.org>]

Prasanth Jayachandran created HIVE-11031:
--------------------------------------------

             Summary: ORC concatenation of old files can fail while merging column statistics
                 Key: HIVE-11031
                 URL: https://issues.apache.org/jira/browse/HIVE-11031
             Project: Hive
          Issue Type: Bug
    Affects Versions: 1.2.0, 1.0.0, 1.1.0, 2.0.0
            Reporter: Prasanth Jayachandran
            Assignee: Prasanth Jayachandran


Column statistics in ORC are optional protobuf fields. Old ORC files might not have statistics for newly added types like decimal, date, timestamp etc. But column statistics merging assumes column statistics exists for these types and invokes merge. For example, merging of TimestampColumnStatistics directly casts the received ColumnStatistics object without doing instanceof check. If the ORC file contains time stamp column statistics then this will work else it will throw ClassCastException.

Also, the file merge operator swallows the exception.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)