You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org> on 2011/12/14 20:49:32 UTC
[jira] [Commented] (GIRAPH-57) Add new RPC call
(putVertexIdMessagesList) to batch putMsgList RPCs together
[ https://issues.apache.org/jira/browse/GIRAPH-57?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169627#comment-13169627 ]
jiraposter@reviews.apache.org commented on GIRAPH-57:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3201/
-----------------------------------------------------------
Review request for giraph.
Summary
-------
Added new putVertexIdMessagesList RPC and supporting classes (VertexIdMessages and VertexIdMessagesList) to reduce the total number of RPCs during a flush. This improves the number of RPC / sec and overall I/O bandwidth. The amount of batching is done by the total number of messages and is configurable at runtime (default of 5000, weighted toward helping smaller messages). I have noted some performance results in https://issues.apache.org/jira/browse/GIRAPH-57 (between 25 - 1075 percent improvements).
Also, while tinkering with BasicRPCCommunications, notices inconsistent spaces between 'synchronized' and '('. Removed spaces and standardized in the CODE_CONVENTIONS.
This addresses bug GIRAPH-57.
https://issues.apache.org/jira/browse/GIRAPH-57
Diffs
-----
http://svn.apache.org/repos/asf/incubator/giraph/trunk/CODE_CONVENTIONS 1213849
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/ArrayListWritable.java 1213849
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1214406
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java 1213849
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/GiraphJob.java 1213849
Diff: https://reviews.apache.org/r/3201/diff
Testing
-------
Passed local and Hadoop unittests. Used the RandomMessageBenchmark on a small cluster.
Thanks,
Avery
> Add new RPC call (putVertexIdMessagesList) to batch putMsgList RPCs together
> ----------------------------------------------------------------------------
>
> Key: GIRAPH-57
> URL: https://issues.apache.org/jira/browse/GIRAPH-57
> Project: Giraph
> Issue Type: Improvement
> Reporter: Jakob Homan
> Assignee: Avery Ching
>
> Right now messages are sent to a vertex one at a time. It would be good to have a putMsgs call that could send messages to multiple vertices (all hosted on the same worker). We'd save a huge number of individual RPC calls at the expense of having smaller calls with larger payloads.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira