You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Maja Kabiljo <ma...@fb.com> on 2012/12/07 18:10:54 UTC
Review Request: Max message request size in bytes,
initialize buffers to expected size
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8407/
-----------------------------------------------------------
Review request for giraph.
Description
-------
Now that all message requests are kept in serialized format, we can limit the size of request by the number of bytes, instead of number of messages in it. Sizing the message buffers to expected size gave about 10% speedup on a large application run.
This addresses bug GIRAPH-445.
https://issues.apache.org/jira/browse/GIRAPH-445
Diffs
-----
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 1417645
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java 1417645
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 1417645
Diff: https://reviews.apache.org/r/8407/diff/
Testing
-------
mvn verify
Large application run on a cluster
Thanks,
Maja Kabiljo
Re: Review Request: Max message request size in bytes,
initialize buffers to expected size
Posted by Maja Kabiljo <ma...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8407/
-----------------------------------------------------------
(Updated Dec. 10, 2012, 6:07 p.m.)
Review request for giraph.
Changes
-------
Avery's comments.
Description
-------
Now that all message requests are kept in serialized format, we can limit the size of request by the number of bytes, instead of number of messages in it. Sizing the message buffers to expected size gave about 10% speedup on a large application run.
This addresses bug GIRAPH-445.
https://issues.apache.org/jira/browse/GIRAPH-445
Diffs (updated)
-----
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 1419615
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java 1419615
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 1419615
Diff: https://reviews.apache.org/r/8407/diff/
Testing
-------
mvn verify
Large application run on a cluster
Thanks,
Maja Kabiljo
Re: Review Request: Max message request size in bytes,
initialize buffers to expected size
Posted by Maja Kabiljo <ma...@fb.com>.
> On Dec. 9, 2012, 9:31 a.m., Avery Ching wrote:
> > +1. This looks pretty good Maja. Just a few minor things belows.
Thanks for the review!
- Maja
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8407/#review14208
-----------------------------------------------------------
On Dec. 10, 2012, 6:07 p.m., Maja Kabiljo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8407/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2012, 6:07 p.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> Now that all message requests are kept in serialized format, we can limit the size of request by the number of bytes, instead of number of messages in it. Sizing the message buffers to expected size gave about 10% speedup on a large application run.
>
>
> This addresses bug GIRAPH-445.
> https://issues.apache.org/jira/browse/GIRAPH-445
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 1419615
> http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java 1419615
> http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 1419615
>
> Diff: https://reviews.apache.org/r/8407/diff/
>
>
> Testing
> -------
>
> mvn verify
> Large application run on a cluster
>
>
> Thanks,
>
> Maja Kabiljo
>
>
Re: Review Request: Max message request size in bytes,
initialize buffers to expected size
Posted by Avery Ching <av...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8407/#review14208
-----------------------------------------------------------
Ship it!
+1. This looks pretty good Maja. Just a few minor things belows.
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java
<https://reviews.apache.org/r/8407/#comment30334>
Maybe MAX_MSG_REQUEST_SIZE and maxMsgRequestSize?
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java
<https://reviews.apache.org/r/8407/#comment30335>
MAX_MSG_SIZE_DEFAULT?
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java
<https://reviews.apache.org/r/8407/#comment30344>
Thanks for getting rid of this.
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java
<https://reviews.apache.org/r/8407/#comment30348>
Size of messages (in bytes) for each worker?
- Avery Ching
On Dec. 7, 2012, 5:10 p.m., Maja Kabiljo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8407/
> -----------------------------------------------------------
>
> (Updated Dec. 7, 2012, 5:10 p.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> Now that all message requests are kept in serialized format, we can limit the size of request by the number of bytes, instead of number of messages in it. Sizing the message buffers to expected size gave about 10% speedup on a large application run.
>
>
> This addresses bug GIRAPH-445.
> https://issues.apache.org/jira/browse/GIRAPH-445
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/GiraphConfiguration.java 1417645
> http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/SendMessageCache.java 1417645
> http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java 1417645
>
> Diff: https://reviews.apache.org/r/8407/diff/
>
>
> Testing
> -------
>
> mvn verify
> Large application run on a cluster
>
>
> Thanks,
>
> Maja Kabiljo
>
>