You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hama.apache.org by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org> on 2012/04/16 18:52:17 UTC

[jira] [Updated] (HAMA-554) Implement a new message manager which writes messages to HDFS rather than sending them over RPC.

     [ https://issues.apache.org/jira/browse/HAMA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Thomas Jungblut updated HAMA-554:
---------------------------------

    Component/s: messaging
                 bsp
         Labels:   (was: features)
    
> Implement a new message manager which writes messages to HDFS rather than sending them over RPC.
> ------------------------------------------------------------------------------------------------
>
>                 Key: HAMA-554
>                 URL: https://issues.apache.org/jira/browse/HAMA-554
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp, messaging
>            Reporter: Apurv Verma
>
> The current message managers send all messages over RPC. In this issue we would like to create a DFSMessageManager which writes all messages to HDFS and store its PATH. Later in the sync() phase these PATHs may be send to other peers which can then read the messages from the HDFS.
> Pros:
> This can handle the case when the number of messages sent in a single superstep are too many to be all contained in memory. HAMA-521 already provides the implementation of Disk based queue. We can abstract this DiskQueue in the util section and use it here.
> Cons:
> Reading data from HDFS might be slow.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira