broadcast variable in spargel

[Martin Neumann <mn...@spotify.com>]

Hej,

I'm using the latest trunk version of Flink and the new JavaAPI.
I'm writing some graph algorithms that need to know the number of nodes in
the graph. To get the number of nodes I run a short count job and then get
a DataSet<Long> that I need to give as input to the other calculations.

It works for normal operations:

nodeList.map(new VertexInit()).withBroadcastSet(numNodes, "#Nodes");


However when I call the Spargelcode I don't get the option to do
.withBroadcastSet

initNodeList.runOperation(VertexCentricIteration.withPlainEdges(edgeList,
new VertexRankUpdater(10, 0.85), new RankMessenger(), 100));


How do I use broadcast variables in Spargel?


cheers Martin

Re: broadcast variable in spargel

[Stephan Ewen <se...@apache.org>]

Hi Martin!

There is an option to use BC variables in Spargel:

The VertexCentricIteration has two methods "
addBroadcastSetForMessagingFunction"and"addBroadcastSetForUpdateFunction".

Stephan



On Mon, Jul 14, 2014 at 1:23 PM, Martin Neumann <mn...@spotify.com>
wrote:

> Hej,
>
> I'm using the latest trunk version of Flink and the new JavaAPI.
> I'm writing some graph algorithms that need to know the number of nodes in
> the graph. To get the number of nodes I run a short count job and then get
> a DataSet<Long> that I need to give as input to the other calculations.
>
> It works for normal operations:
>
> nodeList.map(new VertexInit()).withBroadcastSet(numNodes, "#Nodes");
>
>
> However when I call the Spargelcode I don't get the option to do
> .withBroadcastSet
>
> initNodeList.runOperation(VertexCentricIteration.withPlainEdges(edgeList,
> new VertexRankUpdater(10, 0.85), new RankMessenger(), 100));
>
>
> How do I use broadcast variables in Spargel?
>
>
> cheers Martin
>