You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon (JIRA)" <ji...@apache.org> on 2012/12/14 08:26:13 UTC
[jira] [Issue Comment Deleted] (HAMA-647) Make the input spliter
robustly
[ https://issues.apache.org/jira/browse/HAMA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-647:
--------------------------------
Comment: was deleted
(was: Here's my test patch.)
> Make the input spliter robustly
> --------------------------------
>
> Key: HAMA-647
> URL: https://issues.apache.org/jira/browse/HAMA-647
> Project: Hama
> Issue Type: Improvement
> Components: bsp core
> Affects Versions: 0.5.0, 0.6.0
> Reporter: Yuesheng Hu
> Assignee: Yuesheng Hu
> Priority: Critical
> Labels: patch
> Fix For: 0.6.0
>
> Attachments: commons-module.txt, HAMA-647-2.patch, HAMA-647_3.patch, HAMA-647_4.patch, HAMA-647.patch
>
>
> Currently, the spliter in FileInputFormat is based on the Mapreduce's spliter. But, Hama is different from Mapreduce, Hama's task can not be pended until the slot becomes free. So, the current spliter is not suitable for Hama. When input file is small, it may be ok, but when input is very large, the number of splits will be very large too, even our cluster is powerful enough to handle the input. More details, please see the comments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira