Can a master class control the superstep in Flink Spargel ?

[HungChang <un...@gmail.com>]

Hi,

Would it be available to control the supersteps in Flink Spargel?
For example, a master controls the basic graph algorithm having 5 phases and
the master can switch between the phases.

In the given example of Spargel those are send msg and update msg
sequentially.
Would it be possible to switch to a specific super step?






--
View this message in context: http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Can-a-master-class-control-the-superstep-in-Flink-Spargel-tp733.html
Sent from the Apache Flink (Incubator) User Mailing List archive. mailing list archive at Nabble.com.

Re: Can a master class control the superstep in Flink Spargel ?

[Vasiliki Kalavri <va...@gmail.com>]

Hi,

currently, there is no such built-in master compute class, but you can
easily have the equivalent functionality it as follows:

- If your algorithm has a fixed pattern of superstep types, e.g. an
initialization superstep, a main phase and a finalization superstep, then
you can simply chain these types of Spargel iterations together, by feeding
the output of one as input to the next.

For example:

Graph inputGraph = ...
Graph graph1 = inputGraph.runVertexCentricIteration(new UpdateType1(), new
MessangerType1(), maxIter1);
Graph graph2 = graph1.runVertexCentricIteration(new UpdateType2(), new
MessangerType2(), maxIter2);
Graph graph3 = graph2.runVertexCentricIteration(new UpdateType3(), new
MessangerType3(), maxIter3);

and so on.

- If your algorithm needs to switch between superstep types based on some
control flow that is not defined beforehand, you can emulate the master
functionality by using an iteration aggregator. An aggregator can
accumulate values from all vertices during an iteration and then, the
aggregated value can be read in the beginning of the next iteration. This
way you could keep track of different phases inside your algorithm.

For example, your VertexUpdateFunction could look like this:

VertexUpdater extends VertexUpdateFunction {
int phase;
 @Override
public void preSuperstep() {
phase = this.getIterationAggregator("phase.aggregator");
}

@Override
public void updateVertex(key, value, messages) {
if (phase == 1) {
// do type 1 superstep
....
}
else if (phase ==2) {
// do type 2 superstep
....
}
else ...
}
}

Cheers,
V.

On 14 February 2015 at 22:59, HungChang <un...@gmail.com> wrote:

> Hi,
>
> Would it be available to control the supersteps in Flink Spargel?
> For example, a master controls the basic graph algorithm having 5 phases
> and
> the master can switch between the phases.
>
> In the given example of Spargel those are send msg and update msg
> sequentially.
> Would it be possible to switch to a specific super step?
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-flink-incubator-user-mailing-list-archive.2336050.n4.nabble.com/Can-a-master-class-control-the-superstep-in-Flink-Spargel-tp733.html
> Sent from the Apache Flink (Incubator) User Mailing List archive. mailing
> list archive at Nabble.com.
>