[jira] [Updated] (HUDI-1194) Reorganize HoodieHiveClient and make it fully support Hive Metastore API

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-1194:
---------------------------------
    Labels: pull-request-available  (was: )

> Reorganize HoodieHiveClient and make it fully support Hive Metastore API
> ------------------------------------------------------------------------
>
>                 Key: HUDI-1194
>                 URL: https://issues.apache.org/jira/browse/HUDI-1194
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: Wenning Ding
>            Priority: Major
>              Labels: pull-request-available
>
> Currently there are three ways in HoodieHiveClient to perform Hive functionalities. One is through Hive JDBC, one is through Hive Metastore API. One is through Hive Driver.
>  
>  There’s a parameter called +{{hoodie.datasource.hive_sync.use_jdbc}}+ to control whether use Hive JDBC or not. However, this parameter does not accurately describe the situation.
>  Basically, current logic is when set +*use_jdbc*+ to true, most of the methods in HoodieHiveClient will use JDBC, and few methods in HoodieHiveClient will use Hive Metastore API.
>  When set +*use_jdbc*+ to false, most of the methods in HoodieHiveClient will use Hive Driver, and few methods in HoodieHiveClient will use Hive Metastore API.
> Here is a table shows that what will actually be used when setting use_jdbc to ture/false.
> |Method|use_jdbc=true|use_jdbc=false|
> |{{addPartitionsToTable}}|JDBC|Hive Driver|
> |{{updatePartitionsToTable}}|JDBC|Hive Driver|
> |{{scanTablePartitions}}|Metastore API|Metastore API|
> |{{updateTableDefinition}}|JDBC|Hive Driver|
> |{{createTable}}|JDBC|Hive Driver|
> |{{getTableSchema}}|JDBC|Metastore API|
> |{{doesTableExist}}|Metastore API|Metastore API|
> |getLastCommitTimeSynced|Metastore API|Metastore API|
> [~bschell] and I developed several Metastore API implementation for {{createTable, }}{{addPartitionsToTable}}{{, }}{{updatePartitionsToTable}}{{, }}{{updateTableDefinition }}{{which will be helpful for several issues: e.g. resolving null partition hive sync issue and supporting ALTER_TABLE cascade with AWS glue catalog}}{{. }}
> {{But it seems hard to organize three implementations within the current config. So we plan to separate HoodieHiveClient into three classes:}}
>  # {{HoodieHiveClient which implements all the APIs through Metastore API.}}
>  # {{HoodieHiveJDBCClient which extends from HoodieHiveClient and overwrite several the APIs through Hive JDBC.}}
>  # {{HoodieHiveDriverClient which extends from HoodieHiveClient and overwrite several the APIs through Hive Driver.}}
> {{And we introduce a new parameter }}+*hoodie.datasource.hive_sync.hive_client_class*+ which could** _**_ let you choose which Hive Client class to use.
> {{}}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)