You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Avery Ching <av...@gmail.com> on 2012/10/27 17:14:16 UTC
Review Request: GIRAPH-389 Multithreading should intelligently allocate the
thread pools
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7754/
-----------------------------------------------------------
Review request for giraph.
Description
-------
Do not create more input split loading threads than input splits to allow workers to equally load up input splits
Do not create more compute threads than partitions on the worker
Removed duplicate reserveInputSplit method in BspServiceWorker
Minor optimization to only get the input split zknodes once and pass to all threads.
This addresses bug GIRAPH-389.
https://issues.apache.org/jira/browse/GIRAPH-389
Diffs
-----
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 1402363
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/GraphMapper.java 1402363
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/InputSplitPathOrganizer.java 1402363
http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java 1402363
Diff: https://reviews.apache.org/r/7754/diff/
Testing
-------
passed unittests
ran pagerankbenchmark on a real cluster and observed that the limiting works
Thanks,
Avery Ching