[jira] [Assigned] (SPARK-11131) Worker registration protocol is racy

["Apache Spark (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-11131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-11131:
------------------------------------

    Assignee:     (was: Apache Spark)

> Worker registration protocol is racy
> ------------------------------------
>
>                 Key: SPARK-11131
>                 URL: https://issues.apache.org/jira/browse/SPARK-11131
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.6.0
>            Reporter: Marcelo Vanzin
>            Priority: Minor
>
> I ran into this while making changes to the new RPC framework. Because the Worker registration protocol is based on sending unrelated messages between Master and Worker, it's possible that another message (e.g. caused by an a app trying to allocate workers) to arrive at the Worker before it knows the Master has registered it. This triggers the following code:
> {code}
>     case LaunchExecutor(masterUrl, appId, execId, appDesc, cores_, memory_) =>
>       if (masterUrl != activeMasterUrl) {
>         logWarning("Invalid Master (" + masterUrl + ") attempted to launch executor.")
> {code}
> This may or may not be made worse by SPARK-11098.
> A simple workaround is to use an {{ask}} instead of a {{send}} for these messages. That should at least narrow the race. 
> Note this is more of a problem in {{local-cluster}} mode, used a lot by unit tests, where Master and Worker instances are coming up as part of the app itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org