[jira] [Created] (HUDI-2362) Hudi external configuration file support

["Wenning Ding (Jira)" <ji...@apache.org>]

Wenning Ding created HUDI-2362:
----------------------------------

             Summary: Hudi external configuration file support
                 Key: HUDI-2362
                 URL: https://issues.apache.org/jira/browse/HUDI-2362
             Project: Apache Hudi
          Issue Type: New Feature
            Reporter: Wenning Ding


Many big data applications like Hadoop, Hive have an XML configuration file that users can have a concentrated place to set all the parameters.

Also to support Spark SQL, it might be easier for Hudi to have a configuration file which could avoid setting Hudi parameters inside Hive CLI or Spark SQL CLI.

Here is an example:

{{}}
{code:java}
# Enable optimistic concurrency control by default, to disable it, remove the following two configs
hoodie.write.concurrency.mode           optimistic_concurrency_control
hoodie.cleaner.policy.failed.writes     LAZY

hoodie.write.lock.provider              org.apache.hudi.client.transaction.lock.ZookeeperBasedLockProvider
hoodie.write.lock.zookeeper.url         ip-192-168-1-239.ec2.internal
hoodie.write.lock.zookeeper.port        2181
hoodie.write.lock.zookeeper.base_path   hudi_occ_lock

hoodie.index.type                       BLOOM
# Only applies if index type is HBASE
hoodie.index.hbase.zkquorum             ip-192-168-1-239.ec2.internal
hoodie.index.hbase.zkport               2181

# Only applies if Hive sync is enable
hoodie.datasource.hive_sync.jdbcurl     jdbc:hive2://ip-192-168-1-239.ec2.internal:10000
{code}
{{}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)