You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by jo...@apache.org on 2016/03/23 18:29:45 UTC
spark git commit: [SPARK-14055] writeLocksByTask need to be update
when removeBlock
Repository: spark
Updated Branches:
refs/heads/master 3de24ae2e -> 48ee16d80
[SPARK-14055] writeLocksByTask need to be update when removeBlock
## What changes were proposed in this pull request?
https://issues.apache.org/jira/browse/SPARK-14055
## How was this patch tested?
manual tests by running LiveJournalPageRank on a large dataset ( the dataset must larger enough to incure RDD partition eviction).
Author: Ernest <ea...@gmail.com>
Closes #11875 from Earne/issue-14055.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/48ee16d8
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/48ee16d8
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/48ee16d8
Branch: refs/heads/master
Commit: 48ee16d8012602c75d50aa2a85e26b7de3c48944
Parents: 3de24ae
Author: Ernest <ea...@gmail.com>
Authored: Wed Mar 23 10:29:36 2016 -0700
Committer: Josh Rosen <jo...@databricks.com>
Committed: Wed Mar 23 10:29:36 2016 -0700
----------------------------------------------------------------------
core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/48ee16d8/core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala b/core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala
index 94d11c5..ca53534 100644
--- a/core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala
+++ b/core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala
@@ -421,6 +421,7 @@ private[storage] class BlockInfoManager extends Logging {
infos.remove(blockId)
blockInfo.readerCount = 0
blockInfo.writerTask = BlockInfo.NO_WRITER
+ writeLocksByTask.removeBinding(currentTaskAttemptId, blockId)
}
case None =>
throw new IllegalArgumentException(
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org