You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by mahzad kalantari <ma...@gmail.com> on 2019/11/25 19:03:06 UTC
GraphX performance feedback
Hi all
My question is about GraphX, I 'm looking for user feedbacks on the
performance.
I read this paper written by Facebook team that says Graphx has very poor
performance.
https://engineering.fb.com/core-data/a-comparison-of-state-of-the-art-graph-processing-systems/
Has anyone already encountered performance problems with Graphx, and is it
a good choice if I want to do large scale graph modelling?
Thanks!
Mahzad
Re: GraphX performance feedback
Posted by mahzad kalantari <ma...@gmail.com>.
Ok thanks!
Le jeu. 28 nov. 2019 à 11:27, Phillip Henry <lo...@gmail.com> a
écrit :
> I saw a large improvement in my GraphX processing by:
>
> - using fewer partitions
> - using fewer executors but with much more memory.
>
> YMMV.
>
> Phillip
>
> On Mon, 25 Nov 2019, 19:14 mahzad kalantari, <ma...@gmail.com>
> wrote:
>
>> Thanks for your answer, my use case is friend recommandation for 200
>> million profils.
>>
>> Le lun. 25 nov. 2019 à 14:10, Jörn Franke <jo...@gmail.com> a
>> écrit :
>>
>>> I think it depends what you want do. Interactive big data graph
>>> analytics are probably better of in Janusgraph or similar.
>>> Batch processing (once-off) can be still fine in graphx - you have
>>> though to carefully design the process.
>>>
>>> Am 25.11.2019 um 20:04 schrieb mahzad kalantari <
>>> mahzad.kalantari@gmail.com>:
>>>
>>>
>>> Hi all
>>>
>>> My question is about GraphX, I 'm looking for user feedbacks on the
>>> performance.
>>>
>>> I read this paper written by Facebook team that says Graphx has very
>>> poor performance.
>>>
>>> https://engineering.fb.com/core-data/a-comparison-of-state-of-the-art-graph-processing-systems/
>>>
>>>
>>> Has anyone already encountered performance problems with Graphx, and is
>>> it a good choice if I want to do large scale graph modelling?
>>>
>>>
>>> Thanks!
>>>
>>> Mahzad
>>>
>>>
Re: GraphX performance feedback
Posted by Phillip Henry <lo...@gmail.com>.
I saw a large improvement in my GraphX processing by:
- using fewer partitions
- using fewer executors but with much more memory.
YMMV.
Phillip
On Mon, 25 Nov 2019, 19:14 mahzad kalantari, <ma...@gmail.com>
wrote:
> Thanks for your answer, my use case is friend recommandation for 200
> million profils.
>
> Le lun. 25 nov. 2019 à 14:10, Jörn Franke <jo...@gmail.com> a écrit :
>
>> I think it depends what you want do. Interactive big data graph analytics
>> are probably better of in Janusgraph or similar.
>> Batch processing (once-off) can be still fine in graphx - you have though
>> to carefully design the process.
>>
>> Am 25.11.2019 um 20:04 schrieb mahzad kalantari <
>> mahzad.kalantari@gmail.com>:
>>
>>
>> Hi all
>>
>> My question is about GraphX, I 'm looking for user feedbacks on the
>> performance.
>>
>> I read this paper written by Facebook team that says Graphx has very poor
>> performance.
>>
>> https://engineering.fb.com/core-data/a-comparison-of-state-of-the-art-graph-processing-systems/
>>
>>
>> Has anyone already encountered performance problems with Graphx, and is
>> it a good choice if I want to do large scale graph modelling?
>>
>>
>> Thanks!
>>
>> Mahzad
>>
>>
Re: GraphX performance feedback
Posted by mahzad kalantari <ma...@gmail.com>.
Thanks for your answer, my use case is friend recommandation for 200
million profils.
Le lun. 25 nov. 2019 à 14:10, Jörn Franke <jo...@gmail.com> a écrit :
> I think it depends what you want do. Interactive big data graph analytics
> are probably better of in Janusgraph or similar.
> Batch processing (once-off) can be still fine in graphx - you have though
> to carefully design the process.
>
> Am 25.11.2019 um 20:04 schrieb mahzad kalantari <
> mahzad.kalantari@gmail.com>:
>
>
> Hi all
>
> My question is about GraphX, I 'm looking for user feedbacks on the
> performance.
>
> I read this paper written by Facebook team that says Graphx has very poor
> performance.
>
> https://engineering.fb.com/core-data/a-comparison-of-state-of-the-art-graph-processing-systems/
>
>
> Has anyone already encountered performance problems with Graphx, and is it
> a good choice if I want to do large scale graph modelling?
>
>
> Thanks!
>
> Mahzad
>
>
Re: GraphX performance feedback
Posted by Jörn Franke <jo...@gmail.com>.
I think it depends what you want do. Interactive big data graph analytics are probably better of in Janusgraph or similar.
Batch processing (once-off) can be still fine in graphx - you have though to carefully design the process.
> Am 25.11.2019 um 20:04 schrieb mahzad kalantari <ma...@gmail.com>:
>
>
> Hi all
>
> My question is about GraphX, I 'm looking for user feedbacks on the performance.
>
> I read this paper written by Facebook team that says Graphx has very poor performance.
> https://engineering.fb.com/core-data/a-comparison-of-state-of-the-art-graph-processing-systems/
>
> Has anyone already encountered performance problems with Graphx, and is it a good choice if I want to do large scale graph modelling?
>
>
> Thanks!
>
> Mahzad