You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Hyunsik Choi (JIRA)" <ji...@apache.org> on 2010/03/30 16:31:27 UTC
[jira] Created: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
send method in BSPPeer.java should store messages to local storage temporarily.
--------------------------------------------------------------------------------
Key: HAMA-241
URL: https://issues.apache.org/jira/browse/HAMA-241
Project: Hama
Issue Type: Improvement
Components: bsp
Reporter: Hyunsik Choi
Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon reassigned HAMA-241:
-----------------------------------
Assignee: Edward J. Yoon
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "ChiaHung Lin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ChiaHung Lin updated HAMA-241:
------------------------------
Attachment: hama-241.patch
Remove unnecessary files.
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
> Fix For: 0.2.1
>
> Attachments: hama-241.patch, hama-241.patch
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "ChiaHung Lin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ChiaHung Lin updated HAMA-241:
------------------------------
Status: Patch Available (was: Open)
The file hama-241.patch is for jira issue hama-241. The patch adds features to materialize task data to hdfs and clean those data once tasks are finished.
Also several things relates to this issue, therefore, I would like to get this patch reviewed first and expect more refinement before the patch is accepted.
- BSPPeer
1. Is the purpose of BSPPeer to perform a task? From what I discover so far, it seems to me the BSPPeer is used to perform assigned tasks; and in the future there may have multiple BSPPeers (from wiki Hama architecture) concurrently computing a task. But I do not find any way to obtain task information, which is required to materialize data, from BSPPeer, so at the moment a Task object is passed to BSPPeer for reference.
2. Location of materialize data. Currently data is materialized to hdfs. This considers the issue of rescheduling a task in the future, preventing from problem e.g. media failure which causes the materialized data lost.
- Zookeeper
BSPPeer contains an instance of ZooKeeper, but GroomServer does not. Temporarily solution is to create another zk instance, but in the future the cross cutting issue may need to be addressed?
I appreciate any suggestion.
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
> Fix For: 0.2.1
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-241:
--------------------------------
Affects Version/s: (was: 0.2.1)
0.3.0
Fix Version/s: (was: 0.2.1)
0.3.0
As we planned http://wiki.apache.org/hama/RoadMap, this issue belongs to version 0.3.
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.3.0
> Reporter: Hyunsik Choi
> Assignee: ChiaHung Lin
> Fix For: 0.3.0
>
> Attachments: hama-241.patch, hama-241.patch
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-241:
--------------------------------
Fix Version/s: 0.2.1
Affects Version/s: 0.2.1
move to version of 0.2.1
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
> Fix For: 0.2.1
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874985#action_12874985 ]
Edward J. Yoon commented on HAMA-241:
-------------------------------------
1. In send() mehod, intermediate data should be materialized locally.
2. GroomServer should manage the intermediate data generated during superstep computations.
Then, intermediate data can be used for recovery fails.
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929482#action_12929482 ]
Hudson commented on HAMA-241:
-----------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12459043/hama-241.patch
against trunk revision 1032467.
@author +1. The patch does not contain any @author tags.
tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
patch -1. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hama-Patch/287/console
This message is automatically generated.
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
> Fix For: 0.2.1
>
> Attachments: hama-241.patch
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "ChiaHung Lin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ChiaHung Lin updated HAMA-241:
------------------------------
Attachment: hama-241.patch
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
> Fix For: 0.2.1
>
> Attachments: hama-241.patch
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon reassigned HAMA-241:
-----------------------------------
Assignee: ChiaHung Lin (was: Edward J. Yoon)
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: ChiaHung Lin
> Fix For: 0.2.1
>
> Attachments: hama-241.patch, hama-241.patch
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-241) send method in BSPPeer.java should
store messages to local storage temporarily.
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12929878#action_12929878 ]
Edward J. Yoon commented on HAMA-241:
-------------------------------------
Looks great. I'll look at it.
> send method in BSPPeer.java should store messages to local storage temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
> Fix For: 0.2.1
>
> Attachments: hama-241.patch, hama-241.patch
>
>
> Because of data size and fault tolerance, each BSP peer should store messages into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.