[jira] [Commented] (SPARK-16019) Eliminate unexpected delay during spark on yarn job launch

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-16019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058116#comment-16058116 ] 

Apache Spark commented on SPARK-16019:
--------------------------------------

User 'vanzin' has created a pull request for this issue:
https://github.com/apache/spark/pull/18380

> Eliminate unexpected delay during spark on yarn job launch
> ----------------------------------------------------------
>
>                 Key: SPARK-16019
>                 URL: https://issues.apache.org/jira/browse/SPARK-16019
>             Project: Spark
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: Olasoji
>            Priority: Minor
>
> Currently when launching a job in yarn mode, there will be an added delay of about "spark.yarn.report.interval" seconds before launch. By default this parameter is set to 1 second, however if a user increases this for whatever reason, an unexpected startup delay is introduced.
> The "waitForApplication"  function called during job submission and launch will eventually call monitorApplication and so sleep for the configured interval time before checking the state of the job and proceeding with job startup.
> One solution would be to add an "interval" argument to the monitorApplication function. This added flexibility would allow callers who don't need to wait to set a lower wait interval or none.
> Patch to follow soon



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org