You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Thomas Jungblut (JIRA)" <ji...@apache.org> on 2012/05/08 20:09:48 UTC
[jira] [Created] (HAMA-571) Provide graph repair function in
GraphJobRunner
Thomas Jungblut created HAMA-571:
------------------------------------
Summary: Provide graph repair function in GraphJobRunner
Key: HAMA-571
URL: https://issues.apache.org/jira/browse/HAMA-571
Project: Hama
Issue Type: Improvement
Components: bsp core, graph
Reporter: Thomas Jungblut
As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
If that vertex isn't available it will result in NullPointerExceptions.
Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HAMA-571) Provide graph repair function in
GraphJobRunner
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut resolved HAMA-571.
----------------------------------
Resolution: Fixed
Fix Version/s: 0.5.0
Thanks! I think graph package does not work in distributed mode currently because of aggregators, but I try to fix this in HAMA-575.
> Provide graph repair function in GraphJobRunner
> -----------------------------------------------
>
> Key: HAMA-571
> URL: https://issues.apache.org/jira/browse/HAMA-571
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-571_2.patch, HAMA-571_windows_first.patch
>
>
> As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
> http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
> Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
> If that vertex isn't available it will result in NullPointerExceptions.
> Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-571) Provide graph repair function in
GraphJobRunner
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13276598#comment-13276598 ]
Edward J. Yoon commented on HAMA-571:
-------------------------------------
+1 on adding repair function.
> Provide graph repair function in GraphJobRunner
> -----------------------------------------------
>
> Key: HAMA-571
> URL: https://issues.apache.org/jira/browse/HAMA-571
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-571_windows_first.patch
>
>
> As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
> http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
> Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
> If that vertex isn't available it will result in NullPointerExceptions.
> Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-571) Provide graph repair function in
GraphJobRunner
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-571:
---------------------------------
Attachment: HAMA-571_2.patch
haha, it was a bit more difficult, because we are added an additional sync in the code, therefore some conditions failed (getSuperstepCount > 1).
It is needed to rely on the tracked iteration count, not that sync.
I put that into the testcases of connected components and pagerank. These are two distinct cases, the last one needs self referencing on the edges.
Please review this!
> Provide graph repair function in GraphJobRunner
> -----------------------------------------------
>
> Key: HAMA-571
> URL: https://issues.apache.org/jira/browse/HAMA-571
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-571_2.patch, HAMA-571_windows_first.patch
>
>
> As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
> http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
> Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
> If that vertex isn't available it will result in NullPointerExceptions.
> Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-571) Provide graph repair function in
GraphJobRunner
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277736#comment-13277736 ]
Edward J. Yoon commented on HAMA-571:
-------------------------------------
+1 Let's add this to TRUNK.
> Provide graph repair function in GraphJobRunner
> -----------------------------------------------
>
> Key: HAMA-571
> URL: https://issues.apache.org/jira/browse/HAMA-571
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-571_2.patch, HAMA-571_windows_first.patch
>
>
> As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
> http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
> Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
> If that vertex isn't available it will result in NullPointerExceptions.
> Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HAMA-571) Provide graph repair function in
GraphJobRunner
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut reassigned HAMA-571:
------------------------------------
Assignee: Thomas Jungblut
> Provide graph repair function in GraphJobRunner
> -----------------------------------------------
>
> Key: HAMA-571
> URL: https://issues.apache.org/jira/browse/HAMA-571
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
>
> As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
> http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
> Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
> If that vertex isn't available it will result in NullPointerExceptions.
> Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-571) Provide graph repair function in
GraphJobRunner
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-571:
---------------------------------
Attachment: HAMA-571_windows_first.patch
First scratch in windows and eclipse. I will add testcases tomorrow and generate it without CR's and with real svn.
> Provide graph repair function in GraphJobRunner
> -----------------------------------------------
>
> Key: HAMA-571
> URL: https://issues.apache.org/jira/browse/HAMA-571
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-571_windows_first.patch
>
>
> As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
> http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
> Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
> If that vertex isn't available it will result in NullPointerExceptions.
> Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-571) Provide graph repair function in
GraphJobRunner
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278390#comment-13278390 ]
Hudson commented on HAMA-571:
-----------------------------
Integrated in Hama-Nightly #550 (See [https://builds.apache.org/job/Hama-Nightly/550/])
[HAMA-571]: Provide graph repair function in GraphJobRunner (Revision 1339586)
Result = SUCCESS
tjungblut :
Files :
* /incubator/hama/trunk/CHANGES.txt
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeer.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/util/KeyValuePair.java
* /incubator/hama/trunk/examples/src/test/java/org/apache/hama/examples/MindistSearchTest.java
* /incubator/hama/trunk/examples/src/test/java/org/apache/hama/examples/PageRankTest.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/Edge.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/GraphJobRunner.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/Vertex.java
> Provide graph repair function in GraphJobRunner
> -----------------------------------------------
>
> Key: HAMA-571
> URL: https://issues.apache.org/jira/browse/HAMA-571
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-571_2.patch, HAMA-571_windows_first.patch
>
>
> As discussed here, we want to add a configurable checker that repairs maybe broken graph adjacency lists.
> http://mail-archives.apache.org/mod_mbox/incubator-hama-user/201204.mbox/%3CCAJ-%3DysmSdYoX5t1pPjGi0KXWM%3DYVN2s8CoCgw5V3jXU3F74MHw%40mail.gmail.com%3E
> Currently each vertex addressed in an edge by the adjacency list is needed on the vertex map somewhere in the cluster.
> If that vertex isn't available it will result in NullPointerExceptions.
> Pseudocode is attached in that mail archive.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira