[jira] [Updated] (HUDI-3450) Avoid passing empty string spark master to hudi cli

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-3450:
---------------------------------
    Labels: pull-request-available  (was: )

> Avoid passing empty string spark master to hudi cli
> ---------------------------------------------------
>
>                 Key: HUDI-3450
>                 URL: https://issues.apache.org/jira/browse/HUDI-3450
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Wenning Ding
>            Priority: Major
>              Labels: pull-request-available
>
> When using Hudi CLI, when not passing SparkMaster, by default Hudi CLI should use [SparkUtil.DEFAULT_SPARK_MASTER|https://github.com/apache/hudi/blob/release-0.10.0/hudi-cli/src/main/java/org/apache/hudi/cli/utils/SparkUtil.java#L44]. However, w/ a recent [code change|https://github.com/apache/hudi/commit/445208a0d20b457daeeb5f70995302c92dd19f31] in OSS, when SparkMaster is not passed, it would set Spark master to {{""}} which causes the following exception when initializing a Hudi CLI job:
>  
> {{org.apache.spark.SparkException: Could not parse Master URL: ''at org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2999)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:567)
> at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
> at org.apache.hudi.cli.utils.SparkUtil.initJavaSparkConf(SparkUtil.java:115)
> at org.apache.hudi.cli.utils.SparkUtil.initJavaSparkConf(SparkUtil.java:110)
> at org.apache.hudi.cli.commands.SparkMain.main(SparkMain.java:88)}}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)