You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Reynold Xin <rx...@databricks.com> on 2014/08/14 06:51:42 UTC

proposal for pluggable block transfer interface

Hi devs,

I posted a design doc proposing an interface for pluggable block transfer
(used in shuffle, broadcast, block replication, etc). This is expected to
be done in 1.2 time frame.

It should make our code base cleaner, and enable us to provide alternative
implementations of block transfers (e.g. via the new Netty module that I'm
working on, or possibly via MapR file system).

https://issues.apache.org/jira/browse/SPARK-3019

Please take a look and comment on the JIRA ticket. Thanks.