GraphX: Get edges for a vertex

[Daniil Osipov <da...@shazam.com>]

Hello,

I'm attempting to implement a clustering algorithm on top of Pregel
implementation in GraphX, however I'm hitting a wall. Ideally, I'd like to
be able to get all edges for a specific vertex, since they factor into the
calculation. My understanding was that sendMsg function would receive all
relevant edges in participating vertices (all initially, declining as they
converge and stop changing state), and I was planning to keep vertex edges
associated to each vertex and propagate to other vertices that need to know
about these edges.

What I'm finding is that not all edges get iterated on by sendMsg before
sending messages to vprog. Even if I try to keep track of edges, I don't
account all of them, leading to incorrect results.

The graph I'm testing on has edges between all nodes, one for each
direction, and I'm using EdgeDirection.Both.

Anyone seen something similar, and have some suggestions?
Dan

Re: GraphX: Get edges for a vertex

[Jeffrey Jedele <je...@gmail.com>]

Hi Mas,
I never actually worked with GraphX, but one idea:

As far as I know, you can directly access the vertex and edge RDDs of your
Graph object. Why not simply run a .filter() on the edge RDD to get all
edges that originate from or end at your vertex?

Regards,
Jeff

2015-03-18 10:52 GMT+01:00 mas <ma...@gmail.com>:

> Hi,
>
> Just to continue with the question.
> I need to find the edges of one particular vertex. However,
> (collectNeighbors/collectNeighborIds) provides the neighbors/neighborids
> for
> all the vertices of the graph.
> Any help in this regard will be highly appreciated.
> Thanks,
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-Get-edges-for-a-vertex-tp18880p22115.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: GraphX: Get edges for a vertex

[mas <ma...@gmail.com>]

Hi,

Just to continue with the question. 
I need to find the edges of one particular vertex. However,
(collectNeighbors/collectNeighborIds) provides the neighbors/neighborids for
all the vertices of the graph.
Any help in this regard will be highly appreciated.
Thanks,



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/GraphX-Get-edges-for-a-vertex-tp18880p22115.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: GraphX: Get edges for a vertex

[Takeshi Yamamuro <li...@gmail.com>]

Hi,

I think that there are two solutions;

1. Invalid edges send Iterator.empty messages in sendMsg of the Pregel API.
These messages make no effect on corresponding vertices.

2. Use GraphOps.(collectNeighbors/collectNeighborIds), not the Pregel API
 so as to
handle active edge lists by yourself.
I think that it is hard to handle  edge active lists in Pregel APIs.

Thought?

Best regards,
takeshi

On Fri, Nov 14, 2014 at 7:32 AM, Daniil Osipov <da...@shazam.com>
wrote:

> Hello,
>
> I'm attempting to implement a clustering algorithm on top of Pregel
> implementation in GraphX, however I'm hitting a wall. Ideally, I'd like to
> be able to get all edges for a specific vertex, since they factor into the
> calculation. My understanding was that sendMsg function would receive all
> relevant edges in participating vertices (all initially, declining as they
> converge and stop changing state), and I was planning to keep vertex edges
> associated to each vertex and propagate to other vertices that need to know
> about these edges.
>
> What I'm finding is that not all edges get iterated on by sendMsg before
> sending messages to vprog. Even if I try to keep track of edges, I don't
> account all of them, leading to incorrect results.
>
> The graph I'm testing on has edges between all nodes, one for each
> direction, and I'm using EdgeDirection.Both.
>
> Anyone seen something similar, and have some suggestions?
> Dan
>