You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/12/19 08:33:39 UTC
[GitHub] [spark] cloud-fan commented on a diff in pull request #39041: [SPARK-41528][CONNECT] Merge namespace of Spark Connect and PySpark API
cloud-fan commented on code in PR #39041:
URL: https://github.com/apache/spark/pull/39041#discussion_r1051930672
##########
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:
##########
@@ -229,15 +229,20 @@ private[spark] class SparkSubmit extends Logging {
var childMainClass = ""
// Set the cluster manager
- val clusterManager: Int = args.master match {
- case "yarn" => YARN
- case m if m.startsWith("spark") => STANDALONE
- case m if m.startsWith("mesos") => MESOS
- case m if m.startsWith("k8s") => KUBERNETES
- case m if m.startsWith("local") => LOCAL
- case _ =>
- error("Master must either be yarn or start with spark, mesos, k8s, or local")
- -1
+ val clusterManager: Int = args.maybeMaster match {
+ case Some(v) =>
+ assert(args.maybeRemote.isEmpty)
+ v match {
+ case "yarn" => YARN
+ case m if m.startsWith("spark") => STANDALONE
+ case m if m.startsWith("mesos") => MESOS
+ case m if m.startsWith("k8s") => KUBERNETES
+ case m if m.startsWith("local") => LOCAL
+ case _ =>
+ error("Master must either be yarn or start with spark, mesos, k8s, or local")
+ -1
+ }
+ case None => LOCAL // default master or remote mode.
Review Comment:
do we really have default master? maybe we use a invalid value here, and fail later if `--remove`is not provided either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org