[jira] [Created] (HUDI-515) Resolve API conflict for Hive 2 and Hive 3

["Wenning Ding (Jira)" <ji...@apache.org>]

Wenning Ding created HUDI-515:
---------------------------------

             Summary: Resolve API conflict for Hive 2 and Hive 3
                 Key: HUDI-515
                 URL: https://issues.apache.org/jira/browse/HUDI-515
             Project: Apache Hudi (incubating)
          Issue Type: Sub-task
            Reporter: Wenning Ding


Currently I am working on supporting HIVE 3. There is an API issue.

In [HoodieCombineHiveInputFormat.java|https://code.amazon.com/packages/Aws157Hudi/blobs/a6517d4bfb2584a983342297d350bc9f6d0f38db/--/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/hive/HoodieCombineHiveInputFormat.java], it calls a Hive method: HiveFileFormatUtils.getPartitionDescFromPathRecursively(). But this method is removed in Hive 3 and replaced by HiveFileFormatUtils.getFromPathRecursively().


Ideally, Hudi should support both Hive 2 & Hive 3 so that both
{code:java}
mvn clean install{code}
and
{code:java}
mvn clean install -Dhive.version=3.x{code}
could work. 

One solution is to directly copy source code from [HiveFileFormatUtils.getFromPathRecursively()|https://github.com/apache/hive/blob/release-3.1.2-rc0/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java] and put that code inside [HoodieCombineHiveInputFormat.java|https://code.amazon.com/packages/Aws157Hudi/blobs/a6517d4bfb2584a983342297d350bc9f6d0f38db/--/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/hive/HoodieCombineHiveInputFormat.java].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)