You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by txw <tx...@outlook.com> on 2015/01/15 18:29:50 UTC

Is spark suitable for large scale pagerank, such as 200 million nodes, 2 billion edges?

Hi,


I am run PageRank on a large dataset, which include 200 million nodes and 2 billion edges?
Isspark suitable for large scale pagerank? How many cores and MEM do I need and how long will it take?


Thanks


Xuewei Tang

Re: Is spark suitable for large scale pagerank, such as 200 million nodes, 2 billion edges?

Posted by Ted Yu <yu...@gmail.com>.

Have you seen http://search-hadoop.com/m/JW1q5pE3P12 ?

Please also take a look at the end-to-end performance graph on
http://spark.apache.org/graphx/

Cheers

On Thu, Jan 15, 2015 at 9:29 AM, txw <tx...@outlook.com> wrote:

> Hi,
>
>
> I am run PageRank on a large dataset, which include 200 million nodes and
> 2 billion edges?
>
> Is spark suitable for large scale pagerank? How many cores and MEM do I
> need and how long will it take?
>
>
> Thanks
>
>
> Xuewei Tang
>