You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Thomas Jungblut (JIRA)" <ji...@apache.org> on 2012/05/16 18:35:02 UTC
[jira] [Created] (HAMA-575) Generify graph package
Thomas Jungblut created HAMA-575:
------------------------------------
Summary: Generify graph package
Key: HAMA-575
URL: https://issues.apache.org/jira/browse/HAMA-575
Project: Hama
Issue Type: Improvement
Reporter: Thomas Jungblut
Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278685#comment-13278685 ]
Thomas Jungblut commented on HAMA-575:
--------------------------------------
Actually this should be treated like a process-wide constant in that vertex, which in this case applies the Java conventions of constants which are upper-cased letters.
I will comment on that when committing.
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278680#comment-13278680 ]
Edward J. Yoon commented on HAMA-575:
-------------------------------------
Looks good.
{code}
public static Configuration CONFIGURATION;
{code}
BTW, why did you use uppercase letters?
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-575:
---------------------------------
Component/s: graph
bsp core
Affects Version/s: 0.4.0
Fix Version/s: 0.5.0
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-575:
---------------------------------
Status: Patch Available (was: Open)
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut reassigned HAMA-575:
------------------------------------
Assignee: Thomas Jungblut
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-575:
---------------------------------
Attachment: HAMA-575_3.patch
mindist search works again, just a little to be done now.
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-575:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
got committed, thx!
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278687#comment-13278687 ]
Edward J. Yoon commented on HAMA-575:
-------------------------------------
And, old data seems not available anymore. HAMA-558 patch also should be changed?
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-575:
---------------------------------
Attachment: HAMA-575_2.patch
everything is broken, no motivation currently to make it running -.-
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279293#comment-13279293 ]
Hudson commented on HAMA-575:
-----------------------------
Integrated in Hama-Nightly #551 (See [https://builds.apache.org/job/Hama-Nightly/551/])
[HAMA-575]: Generify graph package (Revision 1340131)
Result = FAILURE
tjungblut :
Files :
* /incubator/hama/trunk/CHANGES.txt
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPJobClient.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/sync/SyncServiceFactory.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/bsp/sync/ZooKeeperSyncClientImpl.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/util/BSPNetUtils.java
* /incubator/hama/trunk/core/src/main/java/org/apache/hama/util/Bytes.java
* /incubator/hama/trunk/core/src/test/java/org/apache/hama/bsp/TestCheckpoint.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/CombineExample.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/ExampleDriver.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/InlinkCount.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/MindistSearch.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/PageRank.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/PiEstimator.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/RandBench.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/SSSP.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/SuperstepPiEstimator.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/util/PagerankTextToSeq.java
* /incubator/hama/trunk/examples/src/main/java/org/apache/hama/examples/util/TextToSequenceFile.java
* /incubator/hama/trunk/examples/src/test/java/org/apache/hama/examples/MindistSearchTest.java
* /incubator/hama/trunk/examples/src/test/java/org/apache/hama/examples/PageRankTest.java
* /incubator/hama/trunk/examples/src/test/java/org/apache/hama/examples/SSSPTest.java
* /incubator/hama/trunk/examples/src/test/java/org/apache/hama/examples/util/PagerankTextToSeqTest.java
* /incubator/hama/trunk/examples/src/test/java/org/apache/hama/examples/util/SSSPTextToSeqTest.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/AverageAggregator.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/Edge.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/GraphJob.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/GraphJobRunner.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/MaxAggregator.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/MinAggregator.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/Vertex.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/VertexInterface.java
* /incubator/hama/trunk/graph/src/main/java/org/apache/hama/graph/VertexWritable.java
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277015#comment-13277015 ]
Thomas Jungblut commented on HAMA-575:
--------------------------------------
TODO:
- GraphJobRunner should be fully generic'd with <VertexWritable, VertexArrayWritable, Text, Writable, Writable>
- VertexInterface's id must be <? extends Writable> instead of hardcoded String
- Edge needs the same ID interface like the vertex and it needs a generic for the value/weight.
That should be enough to make it usable in a more generic fashion.
I propose to add this into 0.5.0 since adding it later will result in mainly incompatible API changes.
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278621#comment-13278621 ]
Thomas Jungblut commented on HAMA-575:
--------------------------------------
I commit this tonight in ~10h if no issues arise. Then I'm going to final optimizations to HAMA-576 and we can move ahead and create a new RC.
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277817#comment-13277817 ]
Thomas Jungblut commented on HAMA-575:
--------------------------------------
It was the parallel test execution by maven...
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13278701#comment-13278701 ]
Thomas Jungblut commented on HAMA-575:
--------------------------------------
Let's close HAMA-558 or make a BSP version out of it. Or we can wait until Suraj is finished with MapReduce and we can translate this to the BSP API.
I just wanted to make it available for others to benchmark.
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Components: bsp core, graph
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Fix For: 0.5.0
>
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-575:
---------------------------------
Attachment: HAMA-575_1.patch
gosh what a refactoring... not yet build, not tested.
I continue tomorrow.
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-575_1.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-575) Generify graph package
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-575:
---------------------------------
Attachment: HAMA-575_final.patch
Deactivated crucial test methods.
This is a very funny observation:
When running each test alone, it works. By running all three tests (no matter parallel or not) they seem to not work.
I guess this is because of the static configuration sharing. But a single testcase is totally enough in my opinion, so I just commented them.
Please check if it also works for you!
My testresult was good:
{noformat}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hama parent POM ............................ SUCCESS [11.754s]
[INFO] core .............................................. SUCCESS [3:41.748s]
[INFO] graph ............................................. SUCCESS [1.175s]
[INFO] examples .......................................... SUCCESS [34.175s]
[INFO] yarn .............................................. SUCCESS [2.599s]
[INFO] hama-dist ......................................... SUCCESS [16.692s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:48.679s
[INFO] Finished at: Thu May 17 16:46:27 CEST 2012
[INFO] Final Memory: 72M/1089M
{noformat}
> Generify graph package
> ----------------------
>
> Key: HAMA-575
> URL: https://issues.apache.org/jira/browse/HAMA-575
> Project: Hama
> Issue Type: Improvement
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
> Attachments: HAMA-575_1.patch, HAMA-575_2.patch, HAMA-575_3.patch, HAMA-575_final.patch
>
>
> Currently many things are hardcoded, we should add generics here too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira