You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by an...@apache.org on 2015/04/14 21:07:13 UTC
spark git commit: [CORE] SPARK-6880: Fixed null check when all the
dependent stages are cancelled due to previous stage failure
Repository: spark
Updated Branches:
refs/heads/master f63b44a5c -> dcf8a9f33
[CORE] SPARK-6880: Fixed null check when all the dependent stages are cancelled due to previous stage failure
Fixed null check when all the dependent stages are cancelled due to previous stage failure. This happens when one of the executor node goes down and all the dependent stages are cancelled.
Author: pankaj arora <pa...@guavus.com>
Closes #5494 from pankajarora12/NEWBRANCH and squashes the following commits:
55ba5e3 [pankaj arora] [CORE] SPARK-6880: Fixed null check when all the dependent stages are cancelled due to previous stage failure
4575720 [pankaj arora] [CORE] SPARK-6880: Fixed null check when all the dependent stages are cancelled due to previous stage failure
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/dcf8a9f3
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/dcf8a9f3
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/dcf8a9f3
Branch: refs/heads/master
Commit: dcf8a9f331c6193a62bbc9282bdc99663e23ca19
Parents: f63b44a
Author: pankaj arora <pa...@guavus.com>
Authored: Tue Apr 14 12:06:46 2015 -0700
Committer: Andrew Or <an...@databricks.com>
Committed: Tue Apr 14 12:07:08 2015 -0700
----------------------------------------------------------------------
.../main/scala/org/apache/spark/scheduler/DAGScheduler.scala | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/dcf8a9f3/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala b/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
index 508fe7b..4a32f89 100644
--- a/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
+++ b/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
@@ -818,12 +818,7 @@ class DAGScheduler(
}
}
- val properties = if (jobIdToActiveJob.contains(jobId)) {
- jobIdToActiveJob(stage.jobId).properties
- } else {
- // this stage will be assigned to "default" pool
- null
- }
+ val properties = jobIdToActiveJob.get(stage.jobId).map(_.properties).orNull
runningStages += stage
// SparkListenerStageSubmitted should be posted before testing whether tasks are
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org