You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2015/05/02 06:23:50 UTC
spark git commit: [SPARK-7317] [Shuffle] Expose shuffle handle
Repository: spark
Updated Branches:
refs/heads/master 38d4e9e44 -> b79aeb95b
[SPARK-7317] [Shuffle] Expose shuffle handle
Details in JIRA, in a nut-shell, all machinary for custom RDD's to leverage spark shuffle directly (without exposing impl details of shuffle) exists - except for this small piece.
Exposing this will allow for custom dependencies to get a handle to ShuffleHandle - which they can then leverage on reduce side.
Author: Mridul Muralidharan <mr...@yahoo-inc.com>
Closes #5857 from mridulm/expose_shuffle_handle and squashes the following commits:
d8b6bd4 [Mridul Muralidharan] Expose ShuffleHandle
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b79aeb95
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b79aeb95
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b79aeb95
Branch: refs/heads/master
Commit: b79aeb95b45ab4ae811039d452cf028d7b844132
Parents: 38d4e9e
Author: Mridul Muralidharan <mr...@yahoo-inc.com>
Authored: Fri May 1 21:23:42 2015 -0700
Committer: Reynold Xin <rx...@databricks.com>
Committed: Fri May 1 21:23:42 2015 -0700
----------------------------------------------------------------------
.../src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/b79aeb95/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala b/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
index 13c7115..e04c97f 100644
--- a/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
+++ b/core/src/main/scala/org/apache/spark/shuffle/ShuffleHandle.scala
@@ -17,9 +17,12 @@
package org.apache.spark.shuffle
+import org.apache.spark.annotation.DeveloperApi
+
/**
* An opaque handle to a shuffle, used by a ShuffleManager to pass information about it to tasks.
*
* @param shuffleId ID of the shuffle
*/
-private[spark] abstract class ShuffleHandle(val shuffleId: Int) extends Serializable {}
+@DeveloperApi
+abstract class ShuffleHandle(val shuffleId: Int) extends Serializable {}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org