[jira] [Resolved] (HAMA-554) Implement a new message manager which writes messages to HDFS rather than sending them over RPC.

["Edward J. Yoon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HAMA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Edward J. Yoon resolved HAMA-554.
---------------------------------

    Resolution: Won't Fix

I don't think this is good idea. I'd rather use local fs and scp than hdfs.

> Implement a new message manager which writes messages to HDFS rather than sending them over RPC.
> ------------------------------------------------------------------------------------------------
>
>                 Key: HAMA-554
>                 URL: https://issues.apache.org/jira/browse/HAMA-554
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp core, messaging
>            Reporter: Apurv Verma
>
> The current message managers send all messages over RPC. In this issue we would like to create a DFSMessageManager which writes all messages to HDFS and store its PATH. Later in the sync() phase these PATHs may be send to other peers which can then read the messages from the HDFS.
> Pros:
> This can handle the case when the number of messages sent in a single superstep are too many to be all contained in memory. HAMA-521 already provides the implementation of Disk based queue. We can abstract this DiskQueue in the util section and use it here.
> Cons:
> Reading data from HDFS might be slow.



--
This message was sent by Atlassian JIRA
(v6.2#6252)