You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Andrew Ash <an...@andrewash.com> on 2014/05/19 09:07:24 UTC

TorrentBroadcast aka Cornet?

Hi Spark devs,

Is the algorithm for
TorrentBroadcast<https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala>the
same as Cornet from the below paper?

http://www.mosharaf.com/wp-content/uploads/orchestra-sigcomm11.pdf

If so it would be nice to include a link to the paper in the Javadoc for
the class.

Thanks!
Andrew

Re: TorrentBroadcast aka Cornet?

Posted by Andrew Ash <an...@andrewash.com>.

Thanks for the info Matei.

Andrew


On Mon, May 19, 2014 at 12:38 AM, Matei Zaharia <ma...@gmail.com>wrote:

> TorrentBroadcast is actually slightly simpler, but it’s based on that. It
> has similar performance. I’d like to make it the default in a future
> version, we just haven’t had a ton of testing with it yet (kind of an
> oversight in this release unfortunately).
>
> Matei
>
> On May 19, 2014, at 12:07 AM, Andrew Ash <an...@andrewash.com> wrote:
>
> > Hi Spark devs,
> >
> > Is the algorithm for
> > TorrentBroadcast<
> https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala
> >the
> > same as Cornet from the below paper?
> >
> > http://www.mosharaf.com/wp-content/uploads/orchestra-sigcomm11.pdf
> >
> > If so it would be nice to include a link to the paper in the Javadoc for
> > the class.
> >
> > Thanks!
> > Andrew
>
>

Re: TorrentBroadcast aka Cornet?

Posted by Matei Zaharia <ma...@gmail.com>.

TorrentBroadcast is actually slightly simpler, but it’s based on that. It has similar performance. I’d like to make it the default in a future version, we just haven’t had a ton of testing with it yet (kind of an oversight in this release unfortunately).

Matei

On May 19, 2014, at 12:07 AM, Andrew Ash <an...@andrewash.com> wrote:

> Hi Spark devs,
> 
> Is the algorithm for
> TorrentBroadcast<https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala>the
> same as Cornet from the below paper?
> 
> http://www.mosharaf.com/wp-content/uploads/orchestra-sigcomm11.pdf
> 
> If so it would be nice to include a link to the paper in the Javadoc for
> the class.
> 
> Thanks!
> Andrew