You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Hyunsik Choi (JIRA)" <ji...@apache.org> on 2009/09/22 13:52:16 UTC
[jira] Created: (HAMA-195) Design the bsp library
Design the bsp library
----------------------
Key: HAMA-195
URL: https://issues.apache.org/jira/browse/HAMA-195
Project: Hama
Issue Type: Task
Components: bsp
Reporter: Hyunsik Choi
Assignee: Hyunsik Choi
First of all, we have to design the bsp library.
1. Analyzing the key requirements of bsp library.
2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764593#action_12764593 ]
Hudson commented on HAMA-195:
-----------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12421720/PATCH-195_3.patch
against trunk revision 824243.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 3 new or modified tests.
javadoc -1. The javadoc tool appears to have generated 1 warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs -1. The patch appears to introduce 5 new Findbugs warnings.
core tests +1. The patch passed core unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/192/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hama-Patch/192/artifact/trunk/build/reports/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/192/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hama-Patch/192/console
This message is automatically generated.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764614#action_12764614 ]
Hudson commented on HAMA-195:
-----------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12421850/HAMA-195_v04.patch
against trunk revision 824243.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 3 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs -1. The patch appears to introduce 3 new Findbugs warnings.
core tests +1. The patch passed core unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/193/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hama-Patch/193/artifact/trunk/build/reports/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hama-Patch/193/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hama-Patch/193/console
This message is automatically generated.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, HAMA-195_v04.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Vaijanath N. Rao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762920#action_12762920 ]
Vaijanath N. Rao commented on HAMA-195:
---------------------------------------
Can we enlist all the required functionalities for the BSP. It will help in designing an interface.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Attachments: HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763333#action_12763333 ]
Edward J. Yoon commented on HAMA-195:
-------------------------------------
To reduce network cost when global synchronization occurs, It would be nice to add some argument for compression in sync() method.
>> it was a Writable. But I modified a Writable to byte [] because it is more general for most of applications.
It is somewhat not clear how these are different. Why not Writable?
Rest are looks great!
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760838#action_12760838 ]
Edward J. Yoon commented on HAMA-195:
-------------------------------------
Good, additionally, it would be nice to write unit tests for showing mechanism of each method and superstep.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Attachments: HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-195:
--------------------------------
Status: Open (was: Patch Available)
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762868#action_12762868 ]
Edward J. Yoon commented on HAMA-195:
-------------------------------------
I've added zookeeper to lib folder. It'll be used for handle the sync barrier.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Attachments: HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764619#action_12764619 ]
Hyunsik Choi commented on HAMA-195:
-----------------------------------
You are welcome. Thank you for accepting.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, HAMA-195_v04.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12762930#action_12762930 ]
Hyunsik Choi commented on HAMA-195:
-----------------------------------
By today night, I will post a diagram that I worked so far and its required functionalities.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Attachments: HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-195:
--------------------------------
Affects Version/s: 0.2.0
Fix Version/s: 0.2.0
editing affect version
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-195:
--------------------------------
Issue Type: New Feature (was: Task)
changing issue type to 'new feature'.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi updated HAMA-195:
------------------------------
Attachment: HAMA-195_2.patch
I implemented the most of layout of bsp, and I attached a patch file including BSPConstants, BSPPeer, DefaultBSPPeer, BSPPeerInterface, and BSPMessage. The basic type for transmission is changed to BSPMessage. In addition, I already tested their send and barrier sync features.
However, I do not post the unit test class because it need a zookeeper server and I cannot afford to consider the unit test with the zookeeper server provided by the hudson. Sooner, I'll upload the improved unit test class.
I ask for reviews of this patch.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi updated HAMA-195:
------------------------------
Attachment: bsplib.uml
bsplib_uml.jpg
I'm going to show works that I have done so far. I attached two files of the uml diagram about the bsp library. One is the image, and another is the staruml file, an open source uml platform. I think that the functional requirements will be well shown in the uml diagram. They have to send and receive in point to point communication, and they have to do barrier sync.
Most of methods follow the conventional bsp library made in C. The main difference is that our one will be based on HadoopRPC and it will use zookeeper in order to control all of the bsp peers.
As shown in the figure, data delivery between peers is carried out by put() method in a RPC interface BSPPeerInterface. The basic unit for transmission is an array of bytes (i.e., byte []). When I posted the interface previously, it was a Writable. But I modified a Writable to byte [] because it is more general for most of applications.
In this design, each tasktracker (i.e., walker in angrapa) will have one BSPPeer instance, and BSPPeers communicates with one another in point to point communication. Also, they enter barrier phase or leave this phase through zookeeper.
I want a few pieces of advice about that.
Thank you for reading.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763784#action_12763784 ]
Edward J. Yoon commented on HAMA-195:
-------------------------------------
Looks great.
BTW, Author tag should be removed.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758980#action_12758980 ]
Hyunsik Choi commented on HAMA-195:
-----------------------------------
I think so. In addition, BSP lib needs two kinds of communication. One is about control communications, and another is for data communications.
The main purpose of control communications is for a master node to control supersteps of slave nodes.
The methods put(), send(), recv() that you mentioned may belong to data communications. Additionally, data communication methods need to support collective communications. For example, send_c() send a set of messages, and recv_r receives a set of messages. The collective communication methods can be improved by some compression methods.
In addition, it would be good to implement the message with Writable interface of Hadoop.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758633#action_12758633 ]
Edward J. Yoon commented on HAMA-195:
-------------------------------------
Basically, BSP should have an remote memory (e.g., put ()) and message-passing (send() and recv ()) capabilities. And, the global communication phase should be happen when all processes are finish on the local by sync(). IMO, rest are could be implemented to real impl class.
WDYT?
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-195:
--------------------------------
Attachment: HAMA-195_v04.patch
added package.html -- 'BSP package of Hama is a framework for developing BSP-style applications'
removed javadoc warnings.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, HAMA-195_v04.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi updated HAMA-195:
------------------------------
Attachment: PATCH-195_3.patch
I attched the more improved patch than previous one. Even though it is still rough, this version includes a test unit for barrier sync on HamaCluster. In addition, I fixed some bugs of BSPPeer class and removed the author tag.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12760198#action_12760198 ]
Edward J. Yoon commented on HAMA-195:
-------------------------------------
Could we follow this standard? BSPlib - The Bulk Synchronous Parallel World-Wide standard library -- http://www.bsp-worldwide.org/implmnts/oxtool/man/bsplib.3.html
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-195:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks Hyunsik.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, HAMA-195_v04.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-195:
--------------------------------
Status: Patch Available (was: Open)
+1 Looks great.
Running patch through hudson.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763336#action_12763336 ]
Hyunsik Choi commented on HAMA-195:
-----------------------------------
Compression is a nice idea. In addition, I plan to make several variations of outgoingQueue. As you know, the outgoing queue is a temporal storage that resides in each peer for transmit data to other peers at the synchronization phase. Each variation will be based on only memory and buffered file respectively.
Now, my thinking about the byte [] is changed to some class like Put of Hbase. It can deal with not only byte [] and Writable.
Sooner, I'll update the uml diagram and interface files for bsp.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758983#action_12758983 ]
Hyunsik Choi commented on HAMA-195:
-----------------------------------
Both BSP master and slaves classes can be implements by two interfaces extended by VersionedProtocol. Also, I think that they can be helped by zookeeper for controlling distributed nodes. Here, we need to consider how to apply zookeeper to BSP control interfaces,
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi updated HAMA-195:
------------------------------
Attachment: HAMA-195.patch
I attached two interface. A BSPPeer interface will be implemented to a RPC client that communicate data with other BSP peers. BSPRPCProtocolVersion is just for the protocol version.
I intended to make them resemble to the original bsplib, but I tried to adapt them for hadoop. Actually, they should be improved by considering how to do barrier sync and design of master.
Sooner, I will post other interfaces including BSPMaster.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Attachments: HAMA-195.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-195) Design the bsp library
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-195:
--------------------------------
Status: Patch Available (was: Open)
about findbugs warning of mutex related with deadlock, it could be addressed later with fault-tolerance design.
Rescheduling.
> Design the bsp library
> ----------------------
>
> Key: HAMA-195
> URL: https://issues.apache.org/jira/browse/HAMA-195
> Project: Hama
> Issue Type: New Feature
> Components: bsp
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
> Attachments: bsplib.uml, bsplib_uml.jpg, HAMA-195.patch, HAMA-195_2.patch, HAMA-195_v04.patch, PATCH-195_3.patch
>
>
> First of all, we have to design the bsp library.
> 1. Analyzing the key requirements of bsp library.
> 2. Designing the bsp core interfaces.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.