BSP realization on Spark

[Ghousia <gh...@gmail.com>]

Hi,

We are trying to implement a BSP model in Spark with the help of GraphX.
One thing I encountered is a Pregel operator in Graph class. But what I
fail to understand is how the Master and Worker needs to be assigned (BSP),
and how barrier synchronization would happen. The pregel operator provides
a way to define a vertex program, but nothing is mentioned about the
barrier synchronization.

Any help in this regard is truly appreciated.

Many Thanks,
Ghousia.

Re: BSP realization on Spark

[Ankur Dave <an...@gmail.com>]

Master/worker assignment and barrier synchronization are handled by Spark,
so Pregel will automatically coordinate the computation, communication, and
synchronization needed to run for the specified number of supersteps (or
until there are no messages sent in a particular superstep).

Ankur <http://www.ankurdave.com/>

Fwd: BSP realization on Spark

[Ghousia <gh...@gmail.com>]

---------- Forwarded message ----------
From: Ghousia <gh...@gmail.com>
Date: Wed, Jun 18, 2014 at 5:41 PM
Subject: BSP realization on Spark
To: user@spark.apache.org


Hi,

We are trying to implement a BSP model in Spark with the help of GraphX.
One thing I encountered is a Pregel operator in Graph class. But what I
fail to understand is how the Master and Worker needs to be assigned (BSP),
and how barrier synchronization would happen. The pregel operator provides
a way to define a vertex program, but nothing is mentioned about the
barrier synchronization.

Any help in this regard is truly appreciated.

Many Thanks,
Ghousia.