You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "MaoYuan Xian (JIRA)" <ji...@apache.org> on 2013/05/10 05:49:15 UTC
[jira] [Created] (HAMA-756) Timing issue and file merging algorithm
in PartitioningRunner make job fail
MaoYuan Xian created HAMA-756:
---------------------------------
Summary: Timing issue and file merging algorithm in PartitioningRunner make job fail
Key: HAMA-756
URL: https://issues.apache.org/jira/browse/HAMA-756
Project: Hama
Issue Type: Bug
Reporter: MaoYuan Xian
Assignee: MaoYuan Xian
There are two major problems in bsp methor of PartitioningRunner may make the partitioning fail:
1. The call to peer.getNumPeers() may trigger the timing issue. In the special situation when some tasks complete the bsp call but some others just enter the "for (FileStatus statu : status)" loop, these remaining task calling to peer.getNumPeers() will trigger the problem.
2. The algorithm of merging the sequence files has the problem: e.g. when desiredNum is 8 and partitioning task number (peer.getNumPeers()) is 6, the part-7 directory can not find the handler to merging it as a file.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Created] (HAMA-756) Timing issue and file merging algorithm in PartitioningRunner make job fail
Posted by Edward <ed...@udanax.org>.
If I remerber correctly, 1) is solved by calling sync().
Sent from my iPhone
On May 10, 2013, at 12:49 PM, "MaoYuan Xian (JIRA)" <ji...@apache.org> wrote:
> trigger the timing issue. In the special situation when some tasks complete the bsp call but some others just enter the "for (FileStatus statu : status)" loop, these remaining task calling to peer.getNumPeers() will trigger the problem.