[jira] [Created] (SPARK-16930) ApplicationMaster's code that waits for SparkContext is race-prone

["Marcelo Vanzin (JIRA)" <ji...@apache.org>]

Marcelo Vanzin created SPARK-16930:
--------------------------------------

             Summary: ApplicationMaster's code that waits for SparkContext is race-prone
                 Key: SPARK-16930
                 URL: https://issues.apache.org/jira/browse/SPARK-16930
             Project: Spark
          Issue Type: Bug
          Components: YARN
            Reporter: Marcelo Vanzin
            Priority: Minor


While taking a look at SPARK-15937 and checking if there's something wrong with the code, I noticed two races that explain the behavior.

Because they're really narrow races, I'm a little wary of declaring them the cause of that bug. Also because the logs posted there don't really explain what went wrong (and don't really look like a SparkContext was run at all).

The races I found are:

- it's possible, but very unlikely, for an application to instantiate a SparkContext and stop it before the AM enters the loop where it checks for the instance.

- it's possible, but very unlikely, for an application to stop the SparkContext after the AM is already waiting for one, has been notified of its creation, but hasn't yet stored the SparkContext reference in a local variable.

I'll fix those and clean up the code a bit in the process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org