You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Andra Lungu <lu...@gmail.com> on 2015/03/19 09:25:26 UTC
[Delta Iterations] The dirty insides(insights)
Hello,
I've used delta iterations several times up until now, but I just realized
that I never fully understood what happens inside. And the documentation
only explains things from a user's perspective. Which is why I could really
use your help :).
Here goes nothing:
In Gelly, Graph.java, there is a nice runVertexCentricIteration(...)
method, which is in fact Spargel's - that disguises a delta iteration. What
I am trying to do is to set the vertex value before running
DataSet<Vertex<K, VV>> newVertices =
verticesWithInDegrees.runOperation(iteration);
The problem is that after this runOperation, the vertex values get reset.
Now, when I looked in VertexCentricIteration.java's createResult().
It's a plain delta iteration that (more or less) looks like this:
final DeltaIteration<Vertex<VertexKey, VertexValue>, Vertex<VertexKey,
VertexValue>> iteration =
this.initialVertices.iterateDelta(this.initialVertices,
this.maximumNumberOfIterations, zeroKeyPos);
................
// configure coGroup update function with name and broadcast variables
updates = updates.name("Vertex State Updates");
for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
updates = updates.withBroadcastSet(e.f1, e.f0);
}
// let the operator know that we preserve the key field
updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
return iteration.closeWith(updates, updates);
The
DataSet<?>
in the for is the problem. Vertex values get reset. Can someone give me a
hint on how to propagate the vertex value throughout the iteration?
Thanks!
Andra
----------------------------------------------------------------------------------------
P.S. Could someone please subscribe me to the @user mailing list? For some
reason, sending the mail to that recipient fails. Thanks!
Re: [Delta Iterations] The dirty insides(insights)
Posted by Robert Metzger <rm...@apache.org>.
Did you send an empty email to user-subscribe@flink.apache.org ? That
should subscribe you.
On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lu...@gmail.com> wrote:
> Hello,
>
> I've used delta iterations several times up until now, but I just realized
> that I never fully understood what happens inside. And the documentation
> only explains things from a user's perspective. Which is why I could really
> use your help :).
>
> Here goes nothing:
> In Gelly, Graph.java, there is a nice runVertexCentricIteration(...)
> method, which is in fact Spargel's - that disguises a delta iteration. What
> I am trying to do is to set the vertex value before running
>
> DataSet<Vertex<K, VV>> newVertices =
> verticesWithInDegrees.runOperation(iteration);
>
> The problem is that after this runOperation, the vertex values get reset.
> Now, when I looked in VertexCentricIteration.java's createResult().
>
> It's a plain delta iteration that (more or less) looks like this:
>
> final DeltaIteration<Vertex<VertexKey, VertexValue>, Vertex<VertexKey,
> VertexValue>> iteration =
> this.initialVertices.iterateDelta(this.initialVertices,
> this.maximumNumberOfIterations, zeroKeyPos);
>
> ................
>
> // configure coGroup update function with name and broadcast variables
> updates = updates.name("Vertex State Updates");
> for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
> updates = updates.withBroadcastSet(e.f1, e.f0);
> }
>
> // let the operator know that we preserve the key field
> updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
>
> return iteration.closeWith(updates, updates);
>
> The
>
> DataSet<?>
>
> in the for is the problem. Vertex values get reset. Can someone give me a
> hint on how to propagate the vertex value throughout the iteration?
>
> Thanks!
> Andra
>
> ----------------------------------------------------------------------------------------
> P.S. Could someone please subscribe me to the @user mailing list? For some
> reason, sending the mail to that recipient fails. Thanks!
>
Re: [Delta Iterations] The dirty insides(insights)
Posted by Stephan Ewen <se...@apache.org>.
You can always have the vertex value being a tuple3 and let the user set
only a part of it (the value).
It would work the same way as with the vertex key - the user function can
see it, but not change it.
On Sat, Mar 21, 2015 at 8:42 PM, Andra Lungu <lu...@gmail.com> wrote:
> Hi Stephan,
>
> You got the idea! I wanted to make the inDeg and outDeg part of the value
> itself(that was one of my first versions), but then I thought it would be
> too annoying for a user who does not want to access the degrees in
> vertexUpdate/ sendMessages to carry them around.
>
> Because in this scenario, when you do setNewVertexValue() you would need a
> new Tuple3(inDeg, outDeg, value). Also, this approach can introduce a ton
> of bugs; imagine that the user can him/herself change the degree values.
> One workaround I see is to make setNewVertexValue update just the third
> value and keep the degrees from the previous iteration. But I am not sure
> that's possible.
>
> If you say that's a good approach I will keep the degrees in the value.
> There seems to be no other way...
>
> Thanks!
> Andra
>
> On Sat, Mar 21, 2015 at 8:26 PM, Stephan Ewen <se...@apache.org> wrote:
>
> > Hi Andra!
> >
> > I am still not 100% sure about the cause of the problem, but here is a
> bit
> > of background on how we model the computation as a delta iteration and
> how
> > you can extend the vertex state. Maybe that helps answer your question.
> >
> > When using delta iterations for vertex-centric iterations, we keep the
> > vertices in the solution set as a two-tuple (vertex key, vertexvalue).
> The
> > key identifies the element in the solution set, so if we add a new
> version
> > of the vertex (same key, different value), the previous entry gets
> > replaced.
> >
> > The result of the iteration is the latest version of the solution set
> after
> > the last superstep.
> >
> > Making the degree part of the vertex value should work (this makes the
> > vertex a nested tuple (key (inDeg, outDeg, value)). Also changing the
> > solution set types to be a 4-tuple (vertex key, inDegree, outDegree,
> value)
> > should work.
> >
> > Greetings,
> > Stephan
> >
> >
> > On Fri, Mar 20, 2015 at 12:00 PM, Andra Lungu <lu...@gmail.com>
> > wrote:
> >
> > > Hi Stephan,
> > >
> > > What I am trying to do, among other things, is to make the vertex's
> > > inDegree and outDegree available in the vertexUpdate and sendMessages
> > > functions via vertex.getInDegrees().
> > > That is almost done:
> https://github.com/andralungu/flink/tree/spargelExt
> > >
> > > Only the problem is that the inDegrees are computed once, per graph
> > before
> > > starting a vertex centric iteration. So, what I decided to do was to
> > store
> > > the inDegree for each vertex within the Vertex object as an attribute.
> So
> > > far so good,
> > >
> > > DataSet<Vertex<K, VV>> newVertices =
> > > verticesWithInDegrees.runOperation(iteration);
> > >
> > > The vertices with in degrees are the initial vertices with the degrees
> > set
> > > as values.
> > > When runOperation() gets called, precisely when
> > >
> > > createResult()
> > >
> > > from VertexCentricIteration gets called, the inDegrees stored are lost.
> > > This happens because basically in each superstep a new DataSet is
> > created.
> > >
> > > I wanted to know how to keep the degrees there. In other words,
> > > how/where/what are the steps for vertex value updates and how to
> include
> > > the degrees there?
> > >
> > > Thanks!
> > > Andra
> > >
> > >
> > >
> > > On Fri, Mar 20, 2015 at 11:28 AM, Stephan Ewen <se...@apache.org>
> wrote:
> > >
> > > > Hi Andra!
> > > >
> > > > I am not sure I am getting exactly what the question is. The code you
> > > > pasted is from the Spargel API - specifically just forwarding
> > registered
> > > > broadcast variables.
> > > >
> > > > What do you mean with "the vertex values get reset" ?
> > > >
> > > > Stephan
> > > >
> > > > PS: The delta iterations are based in this paper:
> > > > http://arxiv.org/pdf/1208.0088.pdf
> > > >
> > > >
> > > > On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lu...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I've used delta iterations several times up until now, but I just
> > > > realized
> > > > > that I never fully understood what happens inside. And the
> > > documentation
> > > > > only explains things from a user's perspective. Which is why I
> could
> > > > really
> > > > > use your help :).
> > > > >
> > > > > Here goes nothing:
> > > > > In Gelly, Graph.java, there is a nice
> runVertexCentricIteration(...)
> > > > > method, which is in fact Spargel's - that disguises a delta
> > iteration.
> > > > What
> > > > > I am trying to do is to set the vertex value before running
> > > > >
> > > > > DataSet<Vertex<K, VV>> newVertices =
> > > > > verticesWithInDegrees.runOperation(iteration);
> > > > >
> > > > > The problem is that after this runOperation, the vertex values get
> > > reset.
> > > > > Now, when I looked in VertexCentricIteration.java's createResult().
> > > > >
> > > > > It's a plain delta iteration that (more or less) looks like this:
> > > > >
> > > > > final DeltaIteration<Vertex<VertexKey, VertexValue>,
> > Vertex<VertexKey,
> > > > > VertexValue>> iteration =
> > > > > this.initialVertices.iterateDelta(this.initialVertices,
> > > > > this.maximumNumberOfIterations, zeroKeyPos);
> > > > >
> > > > > ................
> > > > >
> > > > > // configure coGroup update function with name and broadcast
> > variables
> > > > > updates = updates.name("Vertex State Updates");
> > > > > for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
> > > > > updates = updates.withBroadcastSet(e.f1, e.f0);
> > > > > }
> > > > >
> > > > > // let the operator know that we preserve the key field
> > > > >
> updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
> > > > >
> > > > > return iteration.closeWith(updates, updates);
> > > > >
> > > > > The
> > > > >
> > > > > DataSet<?>
> > > > >
> > > > > in the for is the problem. Vertex values get reset. Can someone
> give
> > > me a
> > > > > hint on how to propagate the vertex value throughout the iteration?
> > > > >
> > > > > Thanks!
> > > > > Andra
> > > > >
> > > > >
> > > >
> > >
> >
> ----------------------------------------------------------------------------------------
> > > > > P.S. Could someone please subscribe me to the @user mailing list?
> For
> > > > some
> > > > > reason, sending the mail to that recipient fails. Thanks!
> > > > >
> > > >
> > >
> >
>
Re: [Delta Iterations] The dirty insides(insights)
Posted by Andra Lungu <lu...@gmail.com>.
Hi Stephan,
You got the idea! I wanted to make the inDeg and outDeg part of the value
itself(that was one of my first versions), but then I thought it would be
too annoying for a user who does not want to access the degrees in
vertexUpdate/ sendMessages to carry them around.
Because in this scenario, when you do setNewVertexValue() you would need a
new Tuple3(inDeg, outDeg, value). Also, this approach can introduce a ton
of bugs; imagine that the user can him/herself change the degree values.
One workaround I see is to make setNewVertexValue update just the third
value and keep the degrees from the previous iteration. But I am not sure
that's possible.
If you say that's a good approach I will keep the degrees in the value.
There seems to be no other way...
Thanks!
Andra
On Sat, Mar 21, 2015 at 8:26 PM, Stephan Ewen <se...@apache.org> wrote:
> Hi Andra!
>
> I am still not 100% sure about the cause of the problem, but here is a bit
> of background on how we model the computation as a delta iteration and how
> you can extend the vertex state. Maybe that helps answer your question.
>
> When using delta iterations for vertex-centric iterations, we keep the
> vertices in the solution set as a two-tuple (vertex key, vertexvalue). The
> key identifies the element in the solution set, so if we add a new version
> of the vertex (same key, different value), the previous entry gets
> replaced.
>
> The result of the iteration is the latest version of the solution set after
> the last superstep.
>
> Making the degree part of the vertex value should work (this makes the
> vertex a nested tuple (key (inDeg, outDeg, value)). Also changing the
> solution set types to be a 4-tuple (vertex key, inDegree, outDegree, value)
> should work.
>
> Greetings,
> Stephan
>
>
> On Fri, Mar 20, 2015 at 12:00 PM, Andra Lungu <lu...@gmail.com>
> wrote:
>
> > Hi Stephan,
> >
> > What I am trying to do, among other things, is to make the vertex's
> > inDegree and outDegree available in the vertexUpdate and sendMessages
> > functions via vertex.getInDegrees().
> > That is almost done: https://github.com/andralungu/flink/tree/spargelExt
> >
> > Only the problem is that the inDegrees are computed once, per graph
> before
> > starting a vertex centric iteration. So, what I decided to do was to
> store
> > the inDegree for each vertex within the Vertex object as an attribute. So
> > far so good,
> >
> > DataSet<Vertex<K, VV>> newVertices =
> > verticesWithInDegrees.runOperation(iteration);
> >
> > The vertices with in degrees are the initial vertices with the degrees
> set
> > as values.
> > When runOperation() gets called, precisely when
> >
> > createResult()
> >
> > from VertexCentricIteration gets called, the inDegrees stored are lost.
> > This happens because basically in each superstep a new DataSet is
> created.
> >
> > I wanted to know how to keep the degrees there. In other words,
> > how/where/what are the steps for vertex value updates and how to include
> > the degrees there?
> >
> > Thanks!
> > Andra
> >
> >
> >
> > On Fri, Mar 20, 2015 at 11:28 AM, Stephan Ewen <se...@apache.org> wrote:
> >
> > > Hi Andra!
> > >
> > > I am not sure I am getting exactly what the question is. The code you
> > > pasted is from the Spargel API - specifically just forwarding
> registered
> > > broadcast variables.
> > >
> > > What do you mean with "the vertex values get reset" ?
> > >
> > > Stephan
> > >
> > > PS: The delta iterations are based in this paper:
> > > http://arxiv.org/pdf/1208.0088.pdf
> > >
> > >
> > > On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lu...@gmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I've used delta iterations several times up until now, but I just
> > > realized
> > > > that I never fully understood what happens inside. And the
> > documentation
> > > > only explains things from a user's perspective. Which is why I could
> > > really
> > > > use your help :).
> > > >
> > > > Here goes nothing:
> > > > In Gelly, Graph.java, there is a nice runVertexCentricIteration(...)
> > > > method, which is in fact Spargel's - that disguises a delta
> iteration.
> > > What
> > > > I am trying to do is to set the vertex value before running
> > > >
> > > > DataSet<Vertex<K, VV>> newVertices =
> > > > verticesWithInDegrees.runOperation(iteration);
> > > >
> > > > The problem is that after this runOperation, the vertex values get
> > reset.
> > > > Now, when I looked in VertexCentricIteration.java's createResult().
> > > >
> > > > It's a plain delta iteration that (more or less) looks like this:
> > > >
> > > > final DeltaIteration<Vertex<VertexKey, VertexValue>,
> Vertex<VertexKey,
> > > > VertexValue>> iteration =
> > > > this.initialVertices.iterateDelta(this.initialVertices,
> > > > this.maximumNumberOfIterations, zeroKeyPos);
> > > >
> > > > ................
> > > >
> > > > // configure coGroup update function with name and broadcast
> variables
> > > > updates = updates.name("Vertex State Updates");
> > > > for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
> > > > updates = updates.withBroadcastSet(e.f1, e.f0);
> > > > }
> > > >
> > > > // let the operator know that we preserve the key field
> > > > updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
> > > >
> > > > return iteration.closeWith(updates, updates);
> > > >
> > > > The
> > > >
> > > > DataSet<?>
> > > >
> > > > in the for is the problem. Vertex values get reset. Can someone give
> > me a
> > > > hint on how to propagate the vertex value throughout the iteration?
> > > >
> > > > Thanks!
> > > > Andra
> > > >
> > > >
> > >
> >
> ----------------------------------------------------------------------------------------
> > > > P.S. Could someone please subscribe me to the @user mailing list? For
> > > some
> > > > reason, sending the mail to that recipient fails. Thanks!
> > > >
> > >
> >
>
Re: [Delta Iterations] The dirty insides(insights)
Posted by Stephan Ewen <se...@apache.org>.
Hi Andra!
I am still not 100% sure about the cause of the problem, but here is a bit
of background on how we model the computation as a delta iteration and how
you can extend the vertex state. Maybe that helps answer your question.
When using delta iterations for vertex-centric iterations, we keep the
vertices in the solution set as a two-tuple (vertex key, vertexvalue). The
key identifies the element in the solution set, so if we add a new version
of the vertex (same key, different value), the previous entry gets replaced.
The result of the iteration is the latest version of the solution set after
the last superstep.
Making the degree part of the vertex value should work (this makes the
vertex a nested tuple (key (inDeg, outDeg, value)). Also changing the
solution set types to be a 4-tuple (vertex key, inDegree, outDegree, value)
should work.
Greetings,
Stephan
On Fri, Mar 20, 2015 at 12:00 PM, Andra Lungu <lu...@gmail.com> wrote:
> Hi Stephan,
>
> What I am trying to do, among other things, is to make the vertex's
> inDegree and outDegree available in the vertexUpdate and sendMessages
> functions via vertex.getInDegrees().
> That is almost done: https://github.com/andralungu/flink/tree/spargelExt
>
> Only the problem is that the inDegrees are computed once, per graph before
> starting a vertex centric iteration. So, what I decided to do was to store
> the inDegree for each vertex within the Vertex object as an attribute. So
> far so good,
>
> DataSet<Vertex<K, VV>> newVertices =
> verticesWithInDegrees.runOperation(iteration);
>
> The vertices with in degrees are the initial vertices with the degrees set
> as values.
> When runOperation() gets called, precisely when
>
> createResult()
>
> from VertexCentricIteration gets called, the inDegrees stored are lost.
> This happens because basically in each superstep a new DataSet is created.
>
> I wanted to know how to keep the degrees there. In other words,
> how/where/what are the steps for vertex value updates and how to include
> the degrees there?
>
> Thanks!
> Andra
>
>
>
> On Fri, Mar 20, 2015 at 11:28 AM, Stephan Ewen <se...@apache.org> wrote:
>
> > Hi Andra!
> >
> > I am not sure I am getting exactly what the question is. The code you
> > pasted is from the Spargel API - specifically just forwarding registered
> > broadcast variables.
> >
> > What do you mean with "the vertex values get reset" ?
> >
> > Stephan
> >
> > PS: The delta iterations are based in this paper:
> > http://arxiv.org/pdf/1208.0088.pdf
> >
> >
> > On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lu...@gmail.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I've used delta iterations several times up until now, but I just
> > realized
> > > that I never fully understood what happens inside. And the
> documentation
> > > only explains things from a user's perspective. Which is why I could
> > really
> > > use your help :).
> > >
> > > Here goes nothing:
> > > In Gelly, Graph.java, there is a nice runVertexCentricIteration(...)
> > > method, which is in fact Spargel's - that disguises a delta iteration.
> > What
> > > I am trying to do is to set the vertex value before running
> > >
> > > DataSet<Vertex<K, VV>> newVertices =
> > > verticesWithInDegrees.runOperation(iteration);
> > >
> > > The problem is that after this runOperation, the vertex values get
> reset.
> > > Now, when I looked in VertexCentricIteration.java's createResult().
> > >
> > > It's a plain delta iteration that (more or less) looks like this:
> > >
> > > final DeltaIteration<Vertex<VertexKey, VertexValue>, Vertex<VertexKey,
> > > VertexValue>> iteration =
> > > this.initialVertices.iterateDelta(this.initialVertices,
> > > this.maximumNumberOfIterations, zeroKeyPos);
> > >
> > > ................
> > >
> > > // configure coGroup update function with name and broadcast variables
> > > updates = updates.name("Vertex State Updates");
> > > for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
> > > updates = updates.withBroadcastSet(e.f1, e.f0);
> > > }
> > >
> > > // let the operator know that we preserve the key field
> > > updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
> > >
> > > return iteration.closeWith(updates, updates);
> > >
> > > The
> > >
> > > DataSet<?>
> > >
> > > in the for is the problem. Vertex values get reset. Can someone give
> me a
> > > hint on how to propagate the vertex value throughout the iteration?
> > >
> > > Thanks!
> > > Andra
> > >
> > >
> >
> ----------------------------------------------------------------------------------------
> > > P.S. Could someone please subscribe me to the @user mailing list? For
> > some
> > > reason, sending the mail to that recipient fails. Thanks!
> > >
> >
>
Re: [Delta Iterations] The dirty insides(insights)
Posted by Andra Lungu <lu...@gmail.com>.
Hi Stephan,
What I am trying to do, among other things, is to make the vertex's
inDegree and outDegree available in the vertexUpdate and sendMessages
functions via vertex.getInDegrees().
That is almost done: https://github.com/andralungu/flink/tree/spargelExt
Only the problem is that the inDegrees are computed once, per graph before
starting a vertex centric iteration. So, what I decided to do was to store
the inDegree for each vertex within the Vertex object as an attribute. So
far so good,
DataSet<Vertex<K, VV>> newVertices =
verticesWithInDegrees.runOperation(iteration);
The vertices with in degrees are the initial vertices with the degrees set
as values.
When runOperation() gets called, precisely when
createResult()
from VertexCentricIteration gets called, the inDegrees stored are lost.
This happens because basically in each superstep a new DataSet is created.
I wanted to know how to keep the degrees there. In other words,
how/where/what are the steps for vertex value updates and how to include
the degrees there?
Thanks!
Andra
On Fri, Mar 20, 2015 at 11:28 AM, Stephan Ewen <se...@apache.org> wrote:
> Hi Andra!
>
> I am not sure I am getting exactly what the question is. The code you
> pasted is from the Spargel API - specifically just forwarding registered
> broadcast variables.
>
> What do you mean with "the vertex values get reset" ?
>
> Stephan
>
> PS: The delta iterations are based in this paper:
> http://arxiv.org/pdf/1208.0088.pdf
>
>
> On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lu...@gmail.com>
> wrote:
>
> > Hello,
> >
> > I've used delta iterations several times up until now, but I just
> realized
> > that I never fully understood what happens inside. And the documentation
> > only explains things from a user's perspective. Which is why I could
> really
> > use your help :).
> >
> > Here goes nothing:
> > In Gelly, Graph.java, there is a nice runVertexCentricIteration(...)
> > method, which is in fact Spargel's - that disguises a delta iteration.
> What
> > I am trying to do is to set the vertex value before running
> >
> > DataSet<Vertex<K, VV>> newVertices =
> > verticesWithInDegrees.runOperation(iteration);
> >
> > The problem is that after this runOperation, the vertex values get reset.
> > Now, when I looked in VertexCentricIteration.java's createResult().
> >
> > It's a plain delta iteration that (more or less) looks like this:
> >
> > final DeltaIteration<Vertex<VertexKey, VertexValue>, Vertex<VertexKey,
> > VertexValue>> iteration =
> > this.initialVertices.iterateDelta(this.initialVertices,
> > this.maximumNumberOfIterations, zeroKeyPos);
> >
> > ................
> >
> > // configure coGroup update function with name and broadcast variables
> > updates = updates.name("Vertex State Updates");
> > for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
> > updates = updates.withBroadcastSet(e.f1, e.f0);
> > }
> >
> > // let the operator know that we preserve the key field
> > updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
> >
> > return iteration.closeWith(updates, updates);
> >
> > The
> >
> > DataSet<?>
> >
> > in the for is the problem. Vertex values get reset. Can someone give me a
> > hint on how to propagate the vertex value throughout the iteration?
> >
> > Thanks!
> > Andra
> >
> >
> ----------------------------------------------------------------------------------------
> > P.S. Could someone please subscribe me to the @user mailing list? For
> some
> > reason, sending the mail to that recipient fails. Thanks!
> >
>
Re: [Delta Iterations] The dirty insides(insights)
Posted by Stephan Ewen <se...@apache.org>.
Hi Andra!
I am not sure I am getting exactly what the question is. The code you
pasted is from the Spargel API - specifically just forwarding registered
broadcast variables.
What do you mean with "the vertex values get reset" ?
Stephan
PS: The delta iterations are based in this paper:
http://arxiv.org/pdf/1208.0088.pdf
On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lu...@gmail.com> wrote:
> Hello,
>
> I've used delta iterations several times up until now, but I just realized
> that I never fully understood what happens inside. And the documentation
> only explains things from a user's perspective. Which is why I could really
> use your help :).
>
> Here goes nothing:
> In Gelly, Graph.java, there is a nice runVertexCentricIteration(...)
> method, which is in fact Spargel's - that disguises a delta iteration. What
> I am trying to do is to set the vertex value before running
>
> DataSet<Vertex<K, VV>> newVertices =
> verticesWithInDegrees.runOperation(iteration);
>
> The problem is that after this runOperation, the vertex values get reset.
> Now, when I looked in VertexCentricIteration.java's createResult().
>
> It's a plain delta iteration that (more or less) looks like this:
>
> final DeltaIteration<Vertex<VertexKey, VertexValue>, Vertex<VertexKey,
> VertexValue>> iteration =
> this.initialVertices.iterateDelta(this.initialVertices,
> this.maximumNumberOfIterations, zeroKeyPos);
>
> ................
>
> // configure coGroup update function with name and broadcast variables
> updates = updates.name("Vertex State Updates");
> for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) {
> updates = updates.withBroadcastSet(e.f1, e.f0);
> }
>
> // let the operator know that we preserve the key field
> updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0");
>
> return iteration.closeWith(updates, updates);
>
> The
>
> DataSet<?>
>
> in the for is the problem. Vertex values get reset. Can someone give me a
> hint on how to propagate the vertex value throughout the iteration?
>
> Thanks!
> Andra
>
> ----------------------------------------------------------------------------------------
> P.S. Could someone please subscribe me to the @user mailing list? For some
> reason, sending the mail to that recipient fails. Thanks!
>