[jira] [Commented] (GIRAPH-314) Implement better message grouping to improve performance in SimpleTriangleClosingVertex

["Maja Kabiljo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/GIRAPH-314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448657#comment-13448657 ] 

Maja Kabiljo commented on GIRAPH-314:
-------------------------------------

So the performance gain here comes from the fact that when you spread the load over time you get to process messages before receiving new ones, and processed messages take less space? Did you try implementing this with a combiner?

A few tiny comments:

- There is one small mistake in IntArrayListWritable.hashCode(), you probably wanted to do:
result = result * 19 + iw.get();
or something similar.

- Maybe add a comment there about what the value of giraph.amortizeMessagingCost exactly means. 

- The javadoc for SimpleTriangleClosingVertex seems incorrect to me, wouldn't A also have B in its list and vice versa? If so, can you please fix the comment while you are there?
                
> Implement better message grouping to improve performance in SimpleTriangleClosingVertex
> ---------------------------------------------------------------------------------------
>
>                 Key: GIRAPH-314
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-314
>             Project: Giraph
>          Issue Type: Improvement
>          Components: examples
>    Affects Versions: 0.2.0
>            Reporter: Eli Reisman
>            Assignee: Eli Reisman
>            Priority: Trivial
>             Fix For: 0.2.0
>
>         Attachments: GIRAPH-314-1.patch, GIRAPH-314-2.patch
>
>
> After running SimpleTriangleClosingVertex at scale I'm thinking the sendMessageToAllEdges() is pretty in the code, but its not a good idea in practice since each vertex V sends degree(V)^2 messages right in the first superset in this algorithm. Could do something with a combiner etc. but just grouping messages by hand at the application level by using IntArrayListWritable again does the trick fine.
> Probably should have just done it this way before, but sendMessageToAllEdges() looked so nice. Sigh. Changed unit tests to reflect this new approach, passes mvn verify and cluster, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira