[GitHub] [hudi] danny0405 commented on a diff in pull request #6566: [HUDI-4766] Fix HoodieFlinkClusteringJob

[GitBox <gi...@apache.org>]

danny0405 commented on code in PR #6566:
URL: https://github.com/apache/hudi/pull/6566#discussion_r961206957


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/clustering/FlinkClusteringConfig.java:
##########
@@ -116,14 +131,57 @@ public class FlinkClusteringConfig extends Configuration {
       description = "Min clustering interval of async clustering service, default 10 minutes")
   public Integer minClusteringIntervalSeconds = 600;
 
+  @Parameter(names = {"--hoodie-conf"}, description = "Any configuration that can be set in the properties file "
+          + "(using the CLI parameter \"--props\") can also be passed command line using this parameter.")
+  public List<String> configs = new ArrayList<>();
+
+  @Parameter(names = {"--props"}, description = "Path to properties file on localfs or dfs, with configurations for "
+          + "hoodie client, schema provider, key generator and data source. For hoodie client props, sane defaults are "
+          + "used, but recommend use to provide basic things like metrics endpoints, hive configs etc. For sources, refer"
+          + "to individual classes, for supported properties.")
+  public String propsFilePath = "";
+
+  /**
+   * Read config from properties file (`--props` option) and cmd line (`--hoodie-conf` option).
+   */
+  public static DFSPropertiesConfiguration readConfig(org.apache.hadoop.conf.Configuration hadoopConfig, Path cfgPath, List<String> overriddenProps) {
+    DFSPropertiesConfiguration conf = new DFSPropertiesConfiguration(hadoopConfig, cfgPath);
+    try {
+      if (!overriddenProps.isEmpty()) {
+        conf.addPropsFromStream(new BufferedReader(new StringReader(String.join("\n", overriddenProps))));
+      }
+    } catch (IOException ioe) {
+      throw new HoodieIOException("Unexpected error adding config overrides", ioe);

Review Comment:
   Can the method in `UtilHelpers` be reused ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org