[jira] [Comment Edited] (SPARK-3732) Yarn Client: Add option to NOT System.exit() at end of main()

["Thomas Graves (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152409#comment-14152409 ] 

Thomas Graves edited comment on SPARK-3732 at 9/29/14 10:19 PM:
----------------------------------------------------------------

 I understand your usecase and need for it, but I think at this point I don't think we want to say we support it without properly addressing the bigger picture.  The only public supported non-deprecated api is via spark-submit script.  This means that we won't guarantee backwards compatibility on it. 

Note that we specifically discussed the Client being public on another PR and it was decided that it isn't officially supported.   The only reason the object was left public was for backwards compatibility with how you used to start spark on yarn with the spark-class script.  




was (Author: tgraves):
 I understand your usecase and need for it, but I think at this point I don't think we want to say we support it without properly addressing the bigger picture.  The only public supported non-deprecated api is via spark-submit script.  This means that we won't guarantee backwards compatibility on it. 

Note that we specifically discussed the Client beyond public on another PR and it was decided that it isn't officially supported.   The only reason the object was left public was for backwards compatibility with how you used to start spark on yarn with the spark-class script.  



> Yarn Client: Add option to NOT System.exit() at end of main()
> -------------------------------------------------------------
>
>                 Key: SPARK-3732
>                 URL: https://issues.apache.org/jira/browse/SPARK-3732
>             Project: Spark
>          Issue Type: Improvement
>    Affects Versions: 1.1.0
>            Reporter: Sotos Matzanas
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> We would like to add the ability to create and submit Spark jobs programmatically via Scala/Java. We have found a way to hack this and submit jobs via Yarn, but since 
> org.apache.spark.deploy.yarn.Client.main()
> exits with either 0 or 1 in the end, this will mean exit of our own program. We would like to add an optional spark conf param to NOT exit at the end of the main



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org