[GitHub] [incubator-hudi] yanghua commented on a change in pull request #1452: [HUDI-740]Fix can not specify the sparkMaster and code clean for SparkUtil

[GitBox <gi...@apache.org>]

yanghua commented on a change in pull request #1452: [HUDI-740]Fix can not specify the sparkMaster and code clean for SparkUtil
URL: https://github.com/apache/incubator-hudi/pull/1452#discussion_r405272257
 
 

 ##########
 File path: hudi-cli/src/main/java/org/apache/hudi/cli/utils/SparkUtil.java
 ##########
 @@ -61,9 +62,14 @@ public static SparkLauncher initLauncher(String propertiesFile) throws URISyntax
   }
 
   public static JavaSparkContext initJavaSparkConf(String name) {
+    return initJavaSparkConf(name, Option.empty(), Option.empty());
+  }
+
+  public static JavaSparkContext initJavaSparkConf(String name, Option<String> master,
+                                                   Option<String> executorMemory) {
     SparkConf sparkConf = new SparkConf().setAppName(name);
 
-    String defMasterFromEnv = sparkConf.getenv("SPARK_MASTER");
+    String defMasterFromEnv = master.orElse(sparkConf.getenv("SPARK_MASTER"));
 
 Review comment:
   +1 to introduce `HoodieCliSparkConfig`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services