[jira] [Resolved] (GIRAPH-1025) Flow-control mechanism improving out-of-core feature

["Hassan Eslami (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GIRAPH-1025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hassan Eslami resolved GIRAPH-1025.
-----------------------------------
    Resolution: Fixed

Addressed in GIRAPH-1027

> Flow-control mechanism improving out-of-core feature
> ----------------------------------------------------
>
>                 Key: GIRAPH-1025
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-1025
>             Project: Giraph
>          Issue Type: Improvement
>          Components: graph
>            Reporter: Hassan Eslami
>            Assignee: Hassan Eslami
>
> Current implementation of out-of-core mechanism may fail due to message flooding in applications without message combiner. In such applications the rate at which messages arrive may be much more than the rate of read/write to disk in OOC mechanism. In this scenario, the OOC mechanism will get OOM as it cannot keep up with storing incoming messages in-memory.
> One might say this problem may also appear in input-superstep as we have input vertices and edges going around as messages, but note that in input superstep the rate at which a worker receive messages (vertices and edges) are almost the same as the rate of reading from disk, and that is almost the same as the rate at which offloading data to disk happens. However, this flow-control mechanism can be generalized for input superstep too. At this point, this generalization does not seem necessary though.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)