You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2014/05/04 01:54:15 UTC

[jira] [Commented] (SPARK-1707) Remove 3 second sleep before starting app on YARN

    [ https://issues.apache.org/jira/browse/SPARK-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988854#comment-13988854 ] 

Sandy Ryza commented on SPARK-1707:
-----------------------------------

[~tgraves] [~mridulm@yahoo-inc.com] any thoughts on this?

> Remove 3 second sleep before starting app on YARN
> -------------------------------------------------
>
>                 Key: SPARK-1707
>                 URL: https://issues.apache.org/jira/browse/SPARK-1707
>             Project: Spark
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>
> After acquiring allocations from YARN and launching containers, Spark currently waits for 3 seconds for executors to connect to the driver.  On Spark standalone, nothing like this happens.
> I'm wondering whether we can just remove this sleep entirely.  Is there a reason I'm missing why YARN is different than standalone in this regard?  At the least we could do something smarter like wait until all executors have registered.



--
This message was sent by Atlassian JIRA
(v6.2#6252)