You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2021/11/01 15:54:00 UTC
[jira] [Assigned] (HUDI-2362) Hudi external configuration file
support
[ https://issues.apache.org/jira/browse/HUDI-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan reassigned HUDI-2362:
-----------------------------------------
Assignee: Wenning Ding
> Hudi external configuration file support
> ----------------------------------------
>
> Key: HUDI-2362
> URL: https://issues.apache.org/jira/browse/HUDI-2362
> Project: Apache Hudi
> Issue Type: New Feature
> Reporter: Wenning Ding
> Assignee: Wenning Ding
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Many big data applications like Hadoop, Hive have an XML configuration file that users can have a concentrated place to set all the parameters.
> Also to support Spark SQL, it might be easier for Hudi to have a configuration file which could avoid setting Hudi parameters inside Hive CLI or Spark SQL CLI.
> Here is an example:
> {{}}
> {code:java}
> # Enable optimistic concurrency control by default, to disable it, remove the following two configs
> hoodie.write.concurrency.mode optimistic_concurrency_control
> hoodie.cleaner.policy.failed.writes LAZY
> hoodie.write.lock.provider org.apache.hudi.client.transaction.lock.ZookeeperBasedLockProvider
> hoodie.write.lock.zookeeper.url ip-192-168-1-239.ec2.internal
> hoodie.write.lock.zookeeper.port 2181
> hoodie.write.lock.zookeeper.base_path hudi_occ_lock
> hoodie.index.type BLOOM
> # Only applies if index type is HBASE
> hoodie.index.hbase.zkquorum ip-192-168-1-239.ec2.internal
> hoodie.index.hbase.zkport 2181
> # Only applies if Hive sync is enable
> hoodie.datasource.hive_sync.jdbcurl jdbc:hive2://ip-192-168-1-239.ec2.internal:10000
> {code}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)