You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2017/07/11 18:27:02 UTC
[jira] [Resolved] (SPARK-16019) Eliminate unexpected delay during
spark on yarn job launch
[ https://issues.apache.org/jira/browse/SPARK-16019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-16019.
------------------------------------
Resolution: Fixed
Assignee: Marcelo Vanzin
Fix Version/s: 2.3.0
> Eliminate unexpected delay during spark on yarn job launch
> ----------------------------------------------------------
>
> Key: SPARK-16019
> URL: https://issues.apache.org/jira/browse/SPARK-16019
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Reporter: Olasoji
> Assignee: Marcelo Vanzin
> Priority: Minor
> Fix For: 2.3.0
>
>
> Currently when launching a job in yarn mode, there will be an added delay of about "spark.yarn.report.interval" seconds before launch. By default this parameter is set to 1 second, however if a user increases this for whatever reason, an unexpected startup delay is introduced.
> The "waitForApplication" function called during job submission and launch will eventually call monitorApplication and so sleep for the configured interval time before checking the state of the job and proceeding with job startup.
> One solution would be to add an "interval" argument to the monitorApplication function. This added flexibility would allow callers who don't need to wait to set a lower wait interval or none.
> Patch to follow soon
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org