You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Edward J. Yoon (Created) (JIRA)" <ji...@apache.org> on 2011/12/07 08:04:40 UTC
[jira] [Created] (HAMA-486) Add unit test to verify correctness of
messages between supersteps
Add unit test to verify correctness of messages between supersteps
------------------------------------------------------------------
Key: HAMA-486
URL: https://issues.apache.org/jira/browse/HAMA-486
Project: Hama
Issue Type: Test
Components: bsp, test
Affects Versions: 0.3.0
Reporter: Edward J. Yoon
Priority: Blocker
Fix For: 0.4.0
Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164888#comment-13164888 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
My simple idea is, ... the message counts can be compared between sender and receiver to check whether all messages are exchanged. So, Peers must not leave barrier until synchronization is complete.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164234#comment-13164234 ]
Thomas Jungblut commented on HAMA-486:
--------------------------------------
This must be related to the queue switching at the end of the sync.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HAMA-486) Add unit test to verify correctness of
messages between supersteps
Posted by "Edward J. Yoon (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon resolved HAMA-486.
---------------------------------
Resolution: Fixed
Committed. Thanks Thomas!
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
> Attachments: HAMA-486.patch
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-486) Add unit test to verify correctness of
messages between supersteps
Posted by "Edward J. Yoon (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-486:
--------------------------------
Description: Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps. (was: Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.)
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-486) Add unit test to verify correctness of
messages between supersteps
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-486:
---------------------------------
Attachment: HAMA-486.patch
This patch should be good for this problem. I refactored abit and made two classes (one test / one bsp) obsolete by putting it into the already existing testcase.
This should be bullet proof now.
BTW the randbench tests seems really verbose, let's set the log outputs to DEBUG.
{noformat}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hama parent POM ............................ SUCCESS [2.257s]
[INFO] Apache Hama Core .................................. SUCCESS [1:53.250s]
[INFO] Apache Hama Graph Package ......................... SUCCESS [2.071s]
[INFO] Apache Hama Examples .............................. SUCCESS [1:40.458s]
[INFO] Apache Hama YARN .................................. SUCCESS [39.646s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
{noformat}
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
> Attachments: HAMA-486.patch
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164263#comment-13164263 ]
Thomas Jungblut commented on HAMA-486:
--------------------------------------
I rework the testcase.
We should emit from each task a local message and a message to another peer, containing the superstep.
This will get writting out with the I/O System.
So this should give us a better test coverage.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165028#comment-13165028 ]
Thomas Jungblut commented on HAMA-486:
--------------------------------------
This is bad. What if we executeclearOutgoingQueues after the leaveBarrier()?
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164867#comment-13164867 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
The problems seems like occurs with slow networks.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164255#comment-13164255 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
I couldn't find the problem yet but ... should we send a message with superstep number?
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165077#comment-13165077 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
Hmm, I'll try like that.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164941#comment-13164941 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
Additionally, I was wanted to add some counter of whole message exchanges among Peers. I'm still think that it will be useful.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Currently, sent messages to the next superstep are appears at (next + 1) superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165821#comment-13165821 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
+1
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
> Attachments: HAMA-486.patch
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165132#comment-13165132 ]
Thomas Jungblut commented on HAMA-486:
--------------------------------------
Seems like we are lucky ;)
Please commit this fix, I try to work on a better unit testing.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165083#comment-13165083 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
Oh good. Works well.
I'm going to test with more large cluster.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-486) Add unit test to verify correctness
of messages between supersteps
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165006#comment-13165006 ]
Edward J. Yoon commented on HAMA-486:
-------------------------------------
Oh, .. clearOutgoingQueues() and put() seems access to localQueueForNextIteration concurrently.
> Add unit test to verify correctness of messages between supersteps
> ------------------------------------------------------------------
>
> Key: HAMA-486
> URL: https://issues.apache.org/jira/browse/HAMA-486
> Project: Hama
> Issue Type: Test
> Components: bsp, test
> Affects Versions: 0.3.0
> Reporter: Edward J. Yoon
> Priority: Blocker
> Fix For: 0.4.0
>
>
> Sometimes, sent messages to the next superstep are appears at (next + 2)th superstep. We need to fix this problem and add unit test to verify correctness of messages between supersteps.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira