[jira] [Issue Comment Edited] (HAMA-411) Support checkpoint based on HDFS

["Thomas Jungblut (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HAMA-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060302#comment-13060302 ] 

Thomas Jungblut edited comment on HAMA-411 at 7/6/11 5:45 AM:
--------------------------------------------------------------

My first idea would be to use an abstract method in BSP (Or we can think of adding this to our new BSPPeer.):

{noformat}
abstract boolean checkpoint(long superStep);
{noformat}

Or we are providing a default implementation which always returns true, but the user can override this with his own logic.

So the user is up to handle in which superstep he want the checkpointing / checkpoint(X) = true. 
This method gets called before a superstep starts.

If true we are going to save all the messages in the queues to disk. 

Additionally we should think of a method in BSP class which is helping the user to save his own computation- for example the tentative pagerank map in PageRank Example. Or the user has to take care of it himself when returning true in the method.


      was (Author: thomas.jungblut):
    My first idea would be to use an abstract method in BSP (Or we can think of adding this to our new BSPPeer.):

{noformat}
abstract boolean checkpoint(long superStep);
{noformat}

So the user is up to handle in which superstep he want the checkpointing / checkpoint(X) = true. 
This method gets called before a superstep starts.

If true we are going to save all the messages in the queues to disk. 

Additionally we should think of a method in BSP class which is helping the user to save his own computation- for example the tentative pagerank map in PageRank Example. Or the user has to take care of it himself when returning true in the method.

  
> Support checkpoint based on HDFS
> --------------------------------
>
>                 Key: HAMA-411
>                 URL: https://issues.apache.org/jira/browse/HAMA-411
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp
>            Reporter: Thomas Jungblut
>
> We need to add checkpointing to Hama to deal with fault in future. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira