You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by James <al...@gmail.com> on 2015/01/05 03:28:04 UTC
Using graphx to calculate average distance of a big graph
Recently we want to use spark to calculate the average shortest path
distance between each reachable pair of nodes in a very big graph.
Is there any one ever try this? We hope to discuss about the problem.
Re: Using graphx to calculate average distance of a big graph
Posted by James <al...@gmail.com>.
We are going to estimate the average distance using [HyperAnf](
http://arxiv.org/abs/1011.5599) on a 100 billion edge graph.
2015-01-07 2:18 GMT+08:00 Ankur Dave <an...@gmail.com>:
> [-dev]
>
> What size of graph are you hoping to run this on? For small graphs where
> materializing the all-pairs shortest path is an option, you could simply
> find the APSP using https://github.com/apache/spark/pull/3619 and then
> take the average distance (apsp.map(_._2.toDouble).mean).
>
> Ankur <http://www.ankurdave.com/>
>
> On Sun, Jan 4, 2015 at 6:28 PM, James <al...@gmail.com> wrote:
>
>> Recently we want to use spark to calculate the average shortest path
>> distance between each reachable pair of nodes in a very big graph.
>>
>> Is there any one ever try this? We hope to discuss about the problem.
>>
>
>
Re: Using graphx to calculate average distance of a big graph
Posted by Ankur Dave <an...@gmail.com>.
[-dev]
What size of graph are you hoping to run this on? For small graphs where
materializing the all-pairs shortest path is an option, you could simply
find the APSP using https://github.com/apache/spark/pull/3619 and then take
the average distance (apsp.map(_._2.toDouble).mean).
Ankur <http://www.ankurdave.com/>
On Sun, Jan 4, 2015 at 6:28 PM, James <al...@gmail.com> wrote:
> Recently we want to use spark to calculate the average shortest path
> distance between each reachable pair of nodes in a very big graph.
>
> Is there any one ever try this? We hope to discuss about the problem.
>