You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xuefu Zhang (JIRA)" <ji...@apache.org> on 2019/07/29 21:52:00 UTC
[jira] [Commented] (FLINK-13438) Hive source/sink/udx should
respect the conversion class of DataType
[ https://issues.apache.org/jira/browse/FLINK-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16895612#comment-16895612 ]
Xuefu Zhang commented on FLINK-13438:
-------------------------------------
Hi [~TsReaper], Thank you for reporting the issue and providing a fix. I think I understand the problem after reading the whole discussion and your patch.
To help others understand the problem: Yes, Hive represents Date/TimeStamp data with classes are in java.sql.*, while Flink expects java.time.LocalDate objects (by default). For this, conversion is needed. However, the current conversion takes an identity conversion.
Since you have already provided a patch here, I'd suggest you convert it to a PR at github as that's how code is contributed in Flink. You might also want to rephrase the description here to better outline the problem. Your fix seems reasonable at my first glance. As critical as it can be, we'd like to include a fix for this in 1.9.
BTW, the new test cases about those types are missing in your patch (possibly in the new file HiveCatalogDataTypeTest.java which is not included).
> Hive source/sink/udx should respect the conversion class of DataType
> --------------------------------------------------------------------
>
> Key: FLINK-13438
> URL: https://issues.apache.org/jira/browse/FLINK-13438
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Hive
> Reporter: Caizhi Weng
> Assignee: Caizhi Weng
> Priority: Critical
> Fix For: 1.9.0, 1.10.0
>
> Attachments: 0001-hive.patch
>
>
> Similar to JDBC connectors, Hive connectors communicate with Flink framework using TableSchema, which contains DataType. As the time data read from and write to Hive connectors must be java.sql.* types and the default conversion class of our time data types are java.time.*, we have to fix Hive connector with DataTypes.DATE/TIME/TIMESTAMP support.
> But currently when reading tables from Hive, the table schema is created using Hive's schema, so the time types in the created schema will be sql time type not local time type. If user specifies a local time type in the table schema when creating a table in Hive, he will get a different schema when reading it out. This is undesired.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)