GraphX Pregel halting condition

[Jay Hutfles <ja...@gmail.com>]

I'm trying to implement a graph algorithm that does a form of path
searching.  Once a certain criteria is met on any path in the graph, I
wanted to halt the rest of the iterations.  But I can't see how to do that
with the Pregel API, since any vertex isn't able to know the state of other
arbitrary vertex (if they're not adjacent).

Is there a common pattern for doing something like this?  I was thinking of
using a custom accumulator where the zero is true and the addInPlace is a
boolean or.  Each vertex (as part of its vprog) could add to the
accumulator, and once a path is found which meets the condition, the
accumulator would then have a value of false.  But since workers can't read
accumulators, I don't see how to use that when knowing whether to iterate
again.  That is, unless I reimplement the Pregel class with the added check
when iterating...

Any suggestions?  Thanks in advance!

Re: GraphX Pregel halting condition

[Ankur Dave <an...@gmail.com>]

There's no built-in support for doing this, so the best option is to copy and modify Pregel to check the accumulator at the end of each iteration. This is robust and shouldn't be too hard, since the Pregel code is short and only uses public GraphX APIs.

Ankur

At 2014-12-03 09:37:01 -0800, Jay Hutfles <ja...@gmail.com> wrote:
> I'm trying to implement a graph algorithm that does a form of path
> searching.  Once a certain criteria is met on any path in the graph, I
> wanted to halt the rest of the iterations.  But I can't see how to do that
> with the Pregel API, since any vertex isn't able to know the state of other
> arbitrary vertex (if they're not adjacent).
>
> Is there a common pattern for doing something like this?  I was thinking of
> using a custom accumulator where the zero is true and the addInPlace is a
> boolean or.  Each vertex (as part of its vprog) could add to the
> accumulator, and once a path is found which meets the condition, the
> accumulator would then have a value of false.  But since workers can't read
> accumulators, I don't see how to use that when knowing whether to iterate
> again.  That is, unless I reimplement the Pregel class with the added check
> when iterating...
>
> Any suggestions?  Thanks in advance!

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org