[jira] [Created] (SPARK-9823) Spark application hangs when using dynamic allocations and executor fails connect to shuffle service

["Thomas Graves (JIRA)" <ji...@apache.org>]

Thomas Graves created SPARK-9823:
------------------------------------

             Summary: Spark application hangs when using dynamic allocations and executor fails connect to shuffle service
                 Key: SPARK-9823
                 URL: https://issues.apache.org/jira/browse/SPARK-9823
             Project: Spark
          Issue Type: Bug
          Components: YARN
    Affects Versions: 1.5.0
            Reporter: Thomas Graves
            Priority: Critical


I was trying out spark 1.5 with the dynamic allocations turned on in YARN.  If the executor fails to connect to the external shuffle service then the application just hangs there.  The SparkContext didn't seem to be doing anything or requesting any new executors to replace it.  

In this case the shuffle service had security on but I forgot to turn it on in the application.

====
This is somewhat a tangent but the reason I tried spark 1.5 is with spark1.4.1 I also found the application hung with no executors so it must not have been requesting them properly.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org