You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Claudio Martella (JIRA)" <ji...@apache.org> on 2013/05/30 01:40:21 UTC
[jira] [Resolved] (GIRAPH-625) DiskBackedMessageStore can merge
fileStores in the background.
[ https://issues.apache.org/jira/browse/GIRAPH-625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claudio Martella resolved GIRAPH-625.
-------------------------------------
Resolution: Duplicate
> DiskBackedMessageStore can merge fileStores in the background.
> --------------------------------------------------------------
>
> Key: GIRAPH-625
> URL: https://issues.apache.org/jira/browse/GIRAPH-625
> Project: Giraph
> Issue Type: Improvement
> Reporter: Claudio Martella
>
> If the number of messages is large compared to the number of messages kept in memory by the DiskBackedMessageStore, it can result in a large number of files. Reading messages for each vertex, requires linearly scanning multiple files at the same time, hence producing a lot of seeks by the disk head.
> While computing the vertices, as the messages for the next superstep flow in, we can keep the number of filestores low by merging them in the background with a thread. The procedure is similar to compaction in NoSQL stores, and the merging of the shuffle & sort of M/R.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira