You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Nishith Agarwal (Jira)" <ji...@apache.org> on 2019/09/26 00:11:00 UTC

[jira] [Comment Edited] (HUDI-180) Move HiveSyncTool registration from hive server (jdbc) to metastore (thrift)

    [ https://issues.apache.org/jira/browse/HUDI-180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16938144#comment-16938144 ] 

Nishith Agarwal edited comment on HUDI-180 at 9/26/19 12:10 AM:
----------------------------------------------------------------

As of now, we did not shed any dependencies but provided support for folks to use metastore API in case a secure (kerberos) type of cluster setup is used.

In the long term, we need to make all the dependencies non-shaded so it can be pulled in from runtime.

The ticket has been merged, closing it now.


was (Author: nishith29):
As of now, we did not shed any dependencies but provided support for folks to use metastore API in case a secure (kerberos) type of cluster setup is used.

The ticket has been merged, closing it now.

> Move HiveSyncTool registration from hive server (jdbc) to metastore (thrift)
> ----------------------------------------------------------------------------
>
>                 Key: HUDI-180
>                 URL: https://issues.apache.org/jira/browse/HUDI-180
>             Project: Apache Hudi (incubating)
>          Issue Type: New Feature
>          Components: Hive Integration
>            Reporter: Nishith Agarwal
>            Assignee: Nishith Agarwal
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently, the HiveSyncTool takes in the JDBC URL of the hive server as an argument to be able to register hoodie datasets with the hive metastore. 
> One of the problems faced was when using HiveSyncTool in a secure HDFS cluster environment using kerberos. The current implementation of JDBC does not allow for registration in such an environment. The implementation can be changed to support that but the consensus internally in our company has been to move to metastore.
> This ticket is to propose this change. Let's discuss on this ticket and I can follow this up with a PR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)