[4/4] git commit: Merge pull request #216 from liancheng/fix-spark-966

[rx...@apache.org]

Merge pull request #216 from liancheng/fix-spark-966

Bugfix: SPARK-965 & SPARK-966

SPARK-965: https://spark-project.atlassian.net/browse/SPARK-965
SPARK-966: https://spark-project.atlassian.net/browse/SPARK-966

* Add back `DAGScheduler.start()`, `eventProcessActor` is created and started here.

  Notice that function is only called by `SparkContext`.

* Cancel the scheduled stage resubmission task when stopping `eventProcessActor`

* Add a new `DAGSchedulerEvent` `ResubmitFailedStages`

  This event message is sent by the scheduled stage resubmission task to `eventProcessActor`.  In this way, `DAGScheduler.resubmitFailedStages()` is guaranteed to be executed from the same thread that runs `DAGScheduler.processEvent()`.

  Please refer to discussion in [SPARK-966](https://spark-project.atlassian.net/browse/SPARK-966) for details.


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/60e23a58
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/60e23a58
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/60e23a58

Branch: refs/heads/master
Commit: 60e23a58b288dae3c87da28e1506323b1d88ee9e
Parents: 743a31a 4a1d966
Author: Reynold Xin <rx...@apache.org>
Authored: Sat Nov 30 23:38:49 2013 -0800
Committer: Reynold Xin <rx...@apache.org>
Committed: Sat Nov 30 23:38:49 2013 -0800

----------------------------------------------------------------------
 .../scala/org/apache/spark/SparkContext.scala   |  1 +
 .../apache/spark/scheduler/DAGScheduler.scala   | 90 ++++++++++++++------
 .../spark/scheduler/DAGSchedulerEvent.scala     |  2 +
 3 files changed, 66 insertions(+), 27 deletions(-)
----------------------------------------------------------------------