You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by vi...@apache.org on 2022/01/06 22:42:56 UTC
[hudi] branch asf-site updated: [HUDI-3076] - Docs for config file details (#4423)
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 1541e99 [HUDI-3076] - Docs for config file details (#4423)
1541e99 is described below
commit 1541e99fa1535a88d8f540174c8f2b21101ececc
Author: Kyle Weller <ky...@gmail.com>
AuthorDate: Thu Jan 6 14:42:04 2022 -0800
[HUDI-3076] - Docs for config file details (#4423)
* added config file details to docs
* Minor cosmetic tweaks
* Minor cosmetic tweaks
Co-authored-by: vinoth chandar <vi...@users.noreply.github.com>
---
website/docs/configurations.md | 6 ++++++
website/versioned_docs/version-0.10.0/configurations.md | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/website/docs/configurations.md b/website/docs/configurations.md
index 02363e7..e2cb35b 100644
--- a/website/docs/configurations.md
+++ b/website/docs/configurations.md
@@ -17,6 +17,12 @@ This page covers the different ways of configuring your job to write/read Hudi t
- [**Kafka Connect Configs**](#KAFKA_CONNECT): These set of configs are used for Kafka Connect Sink Connector for writing Hudi Tables
- [**Amazon Web Services Configs**](#AWS): Please fill in the description for Config Group Name: Amazon Web Services Configs
+## Externalized Config File
+Instead of directly passing configuration settings to every Hudi job, you can also centrally set them in a configuration
+file `hudi-default.conf`. By default, Hudi would load the configuration file under `/etc/hudi/conf` directory. You can
+specify a different configuration directory location by setting the `HUDI_CONF_DIR` environment variable. This can be
+useful for uniformly enforcing repeated configs (like Hive sync or write/index tuning), across your entire data lake.
+
## Spark Datasource Configs {#SPARK_DATASOURCE}
These configs control the Hudi Spark Datasource, providing ability to define keys/partitioning, pick out the write operation, specify how to merge records or choosing query type to read.
diff --git a/website/versioned_docs/version-0.10.0/configurations.md b/website/versioned_docs/version-0.10.0/configurations.md
index 02363e7..4521ce7 100644
--- a/website/versioned_docs/version-0.10.0/configurations.md
+++ b/website/versioned_docs/version-0.10.0/configurations.md
@@ -17,6 +17,12 @@ This page covers the different ways of configuring your job to write/read Hudi t
- [**Kafka Connect Configs**](#KAFKA_CONNECT): These set of configs are used for Kafka Connect Sink Connector for writing Hudi Tables
- [**Amazon Web Services Configs**](#AWS): Please fill in the description for Config Group Name: Amazon Web Services Configs
+## Externalized Config File
+Instead of directly passing configuration settings to every Hudi job, you can also centrally set them in a configuration
+file `hudi-default.conf`. By default, Hudi would load the configuration file under `/etc/hudi/conf` directory. You can
+specify a different configuration directory location by setting the `HUDI_CONF_DIR` environment variable. This can be
+useful for uniformly enforcing repeated configs (like Hive sync or write/index tuning), across your entire data lake.
+
## Spark Datasource Configs {#SPARK_DATASOURCE}
These configs control the Hudi Spark Datasource, providing ability to define keys/partitioning, pick out the write operation, specify how to merge records or choosing query type to read.