You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Avery Ching (JIRA)" <ji...@apache.org> on 2012/10/27 17:17:11 UTC
[jira] [Commented] (GIRAPH-389) Multithreading should intelligently
allocate the thread pools
[ https://issues.apache.org/jira/browse/GIRAPH-389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485451#comment-13485451 ]
Avery Ching commented on GIRAPH-389:
------------------------------------
https://reviews.apache.org/r/7754/
Description:
Do not create more input split loading threads than input splits to allow workers to equally load up input splits
Do not create more compute threads than partitions on the worker
Removed duplicate reserveInputSplit method in BspServiceWorker
Minor optimization to only get the input split zknodes once and pass to all threads.
Testing Done:
passed unittests
ran pagerankbenchmark on a real cluster and observed that the limiting works
> Multithreading should intelligently allocate the thread pools
> -------------------------------------------------------------
>
> Key: GIRAPH-389
> URL: https://issues.apache.org/jira/browse/GIRAPH-389
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
>
> Even if the user suggests a very high number of threads, the input split threads should not exceed the number splits divided by the number of workers.
> The number of compute threads should not be greater than the number of partitions on that worker.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira