You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Samuel Soubeyran (JIRA)" <ji...@apache.org> on 2017/03/25 19:22:41 UTC
[jira] [Commented] (SPARK-15665) spark-submit --kill and --status
are not working
[ https://issues.apache.org/jira/browse/SPARK-15665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15941869#comment-15941869 ]
Samuel Soubeyran commented on SPARK-15665:
------------------------------------------
When trying to do the same thing using the SparkLauncher class, it doesn't work:
new SparkLauncher(sparkEnvMap).setSparkHome(sparkHome).addSparkArg("--kill", submissionId).launch().
This is because SparkLauncher call the empty constructor in SparkSubmitCommandBuilder, bypassing all the logic to handle the kill and status command present in the constructor SparkSubmitCommandBuilder(List<String> args).
I also want to make the case that this logic shouldn't be present in the constructor in the first place since it's a builder pattern. Instead it should be in the buildSparkSubmitCommand. The default should be the empty constructor, the full constructor is just a shortcut. Else what the point of having a builder pattern in the first place.
I'd be happy to send a PR to solve this,
Also there might be a easier way to kill/get the status of a job (cluster mode) but i couldn't figure it out.
Thanks,
Sam
> spark-submit --kill and --status are not working
> -------------------------------------------------
>
> Key: SPARK-15665
> URL: https://issues.apache.org/jira/browse/SPARK-15665
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Devaraj K
> Assignee: Devaraj K
> Fix For: 2.0.0
>
>
> {code:xml}
> [devaraj@server2 spark-master]$ ./bin/spark-submit --kill driver-20160531171222-0000 --master spark://xx.xx.xx.xx:6066
> Exception in thread "main" java.lang.IllegalArgumentException: Missing application resource.
> at org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
> at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitArgs(SparkSubmitCommandBuilder.java:160)
> at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitCommand(SparkSubmitCommandBuilder.java:276)
> at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:151)
> at org.apache.spark.launcher.Main.main(Main.java:86)
> {code}
> {code:xml}
> [devaraj@server2 spark-master]$ ./bin/spark-submit --status driver-20160531171222-0000 --master spark://xx.xx.xx.xx:6066
> Exception in thread "main" java.lang.IllegalArgumentException: Missing application resource.
> at org.apache.spark.launcher.CommandBuilderUtils.checkArgument(CommandBuilderUtils.java:241)
> at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitArgs(SparkSubmitCommandBuilder.java:160)
> at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildSparkSubmitCommand(SparkSubmitCommandBuilder.java:276)
> at org.apache.spark.launcher.SparkSubmitCommandBuilder.buildCommand(SparkSubmitCommandBuilder.java:151)
> at org.apache.spark.launcher.Main.main(Main.java:86)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org