[jira] [Commented] (HIVE-22224) Support Parquet-Avro Timestamp Type

["Brandon Scheller (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961541#comment-16961541 ] 

Brandon Scheller commented on HIVE-22224:
-----------------------------------------

While looking into this, I have found an issue that makes this more complex than converting the Long to Timestamp. Because Hive uses an older version of avro and parquet that does not yet support logical type, we have no way of knowing the precision of the long in relation to timestamp. Spark can write both timestamp_millis and timestamp_micros and hive would have to guess at the precision of the number to convert to timestamp.

> Support Parquet-Avro Timestamp Type
> -----------------------------------
>
>                 Key: HIVE-22224
>                 URL: https://issues.apache.org/jira/browse/HIVE-22224
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema
>    Affects Versions: 2.3.5, 2.3.6
>            Reporter: cdmikechen
>            Assignee: cdmikechen
>            Priority: Major
>              Labels: parquet
>             Fix For: 2.3.7
>
>
> When user create an external table and import a parquet-avro data with 1.8.2 version which supported logical_type in Hive2.3 or before version, Hive can not read timestamp type column data correctly.
> Hive will read it as LongWritable which it actually stores as long(logical_type=timestamp-millis).So we may add some codes in org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableTimestampObjectInspector.java to let Hive cast long type to timestamp type.
> Some code like below:
>  
> public Timestamp getPrimitiveJavaObject(Object o) {
>   if (o instanceof LongWritable) {
>     return new Timestamp(((LongWritable) o).get());
>   }
>   return o == null ? null : ((TimestampWritable) o).getTimestamp();
> }
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)