[jira] [Comment Edited] (HAMA-700) BSPPartitioner should be configurable to be optional and allow input format conversion

["Suraj Menon (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HAMA-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13546932#comment-13546932 ] 

Suraj Menon edited comment on HAMA-700 at 1/8/13 3:07 PM:
----------------------------------------------------------

Looking at some discussions in the mailing list ( http://mail-archives.apache.org/mod_mbox/hama-dev/201301.mbox/%3CCA%2B5TnhoMgP0_PNf2Bh%2BeDGc39u6%3DSXWiFk2O%3D%2BwufisCtQW5Hg%40mail.gmail.com%3E) , I am attaching the current state of patch. Please let me know if the changes till now are in the right track.  I was able to get testSubmitGraph run with partitioning in the unit tests. I am working on the remaining. I moved the partitioner job invocation to partition function. I don't think I can get to run the hama-0.5.0.examples to run with 0.6.1 binaries, but we can convert and do run-time partitioning with the same VertexReader. This would help if some of our users have implemented their own vertex readers. Please let me know if there is any problem with the direction taken. I shall upload the final patch after all unit tests pass.
                
      was (Author: surajsmenon):
    Looking at some discussions in the mailing list ( http://mail-archives.apache.org/mod_mbox/hama-dev/201301.mbox/%3CCA%2B5TnhoMgP0_PNf2Bh%2BeDGc39u6%3DSXWiFk2O%3D%2BwufisCtQW5Hg%40mail.gmail.com%3E) , I am attaching the current state of patch. Please let me know if the changes till now are in the right track.  I was able to get testSubmitGraph run with partitioning in the unit tests. I am working on the remaining. I moved the partitioner job invocation to partition function. I don't think I can get to run the hama-0.5.0.examples to run with 0.6.1 binaries, but we can convert and do run-time partitioning with the same VertexReader. This would help if some of our users have implemented their own vertex readers. Please let me know if there is any problem with the direction taken.
                  
> BSPPartitioner should be configurable to be optional and allow input format conversion
> --------------------------------------------------------------------------------------
>
>                 Key: HAMA-700
>                 URL: https://issues.apache.org/jira/browse/HAMA-700
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp core
>            Reporter: Suraj Menon
>            Assignee: Suraj Menon
>             Fix For: 0.6.1
>
>         Attachments: HAMA-700.patch_Jan7, HAMA-700-v1.patch
>
>
> There should be a provisioning for skipping the PartitionRunner if needed. Also we can have a RecordConverter interface so that the PartitionRunner can read the input in any format and create new splits. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira