You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Suijian Zhou <su...@gmail.com> on 2014/04/02 17:36:37 UTC
Graph stats in giraph.
Hi,
Does anybody know what "Aggregate sent message message bytes" means in
the giraph log file? I'm processing a ~500MB size of graph with 4 worker
nodes. While I see "Aggregate sent message message bytes=33120305280" in
the output, would a ~500MB graph generate ~30GB messages communication on 4
nodes? Thanks!
Best Regards,
Suijian
Re: Graph stats in giraph.
Posted by Suijian Zhou <su...@gmail.com>.
Hi, Lukas,
Thanks for your info! To my understand, these sent messages are all
across machines, i.e. they are communications among different machines?
Best Regards,
Suijian
2014-04-02 10:39 GMT-05:00 Lukas Nalezenec <lu...@firma.seznam.cz>
:
> Hi,
>
> IMHO its sum of bytes sent in messages in all iterations.
> It is possible, it depends on number of edges in your graph, size of
> messages and number of iterations.
>
> Regards,
> Lukas
>
>
> On 2.4.2014 17:36, Suijian Zhou wrote:
>
> Hi,
> Does anybody know what "Aggregate sent message message bytes" means in
> the giraph log file? I'm processing a ~500MB size of graph with 4 worker
> nodes. While I see "Aggregate sent message message bytes=33120305280" in
> the output, would a ~500MB graph generate ~30GB messages communication on 4
> nodes? Thanks!
>
> Best Regards,
> Suijian
>
>
>
Re: Graph stats in giraph.
Posted by Lukas Nalezenec <lu...@firma.seznam.cz>.
Hi,
IMHO its sum of bytes sent in messages in all iterations.
It is possible, it depends on number of edges in your graph, size of
messages and number of iterations.
Regards,
Lukas
On 2.4.2014 17:36, Suijian Zhou wrote:
> Hi,
> Does anybody know what "Aggregate sent message message bytes" means
> in the giraph log file? I'm processing a ~500MB size of graph with 4
> worker nodes. While I see "Aggregate sent message message
> bytes=33120305280" in the output, would a ~500MB graph generate ~30GB
> messages communication on 4 nodes? Thanks!
>
> Best Regards,
> Suijian
Re: Graph stats in giraph.
Posted by Lukas Nalezenec <lu...@firma.seznam.cz>.
Hi,
If you want to reduce the number, you can replace hash partitioning with
range partitioning. You can also add some cache to your computation and
combine messages.
Regards,
Lukas
On 2.4.2014 17:36, Suijian Zhou wrote:
> Hi,
> Does anybody know what "Aggregate sent message message bytes" means
> in the giraph log file? I'm processing a ~500MB size of graph with 4
> worker nodes. While I see "Aggregate sent message message
> bytes=33120305280" in the output, would a ~500MB graph generate ~30GB
> messages communication on 4 nodes? Thanks!
>
> Best Regards,
> Suijian