[jira] [Resolved] (SPARK-16930) ApplicationMaster's code that waits for SparkContext is race-prone

["Marcelo Vanzin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Marcelo Vanzin resolved SPARK-16930.
------------------------------------
       Resolution: Fixed
         Assignee: Marcelo Vanzin
    Fix Version/s: 2.1.0

> ApplicationMaster's code that waits for SparkContext is race-prone
> ------------------------------------------------------------------
>
>                 Key: SPARK-16930
>                 URL: https://issues.apache.org/jira/browse/SPARK-16930
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>            Reporter: Marcelo Vanzin
>            Assignee: Marcelo Vanzin
>            Priority: Minor
>             Fix For: 2.1.0
>
>
> While taking a look at SPARK-15937 and checking if there's something wrong with the code, I noticed two races that explain the behavior.
> Because they're really narrow races, I'm a little wary of declaring them the cause of that bug. Also because the logs posted there don't really explain what went wrong (and don't really look like a SparkContext was run at all).
> The races I found are:
> - it's possible, but very unlikely, for an application to instantiate a SparkContext and stop it before the AM enters the loop where it checks for the instance.
> - it's possible, but very unlikely, for an application to stop the SparkContext after the AM is already waiting for one, has been notified of its creation, but hasn't yet stored the SparkContext reference in a local variable.
> I'll fix those and clean up the code a bit in the process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org