You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Eli Reisman (JIRA)" <ji...@apache.org> on 2012/06/30 22:32:43 UTC
[jira] [Created] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Eli Reisman created GIRAPH-228:
----------------------------------
Summary: SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
Key: GIRAPH-228
URL: https://issues.apache.org/jira/browse/GIRAPH-228
Project: Giraph
Issue Type: Bug
Components: examples, test
Affects Versions: 0.2.0
Reporter: Eli Reisman
Assignee: Eli Reisman
Priority: Trivial
Fix For: 0.2.0
Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429524#comment-13429524 ]
Hudson commented on GIRAPH-228:
-------------------------------
Integrated in Giraph-trunk-Commit #157 (See [https://builds.apache.org/job/Giraph-trunk-Commit/157/])
GIRAPH-228. SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value. Contributed by Eli Reisman. (Revision 1370047)
Result = SUCCESS
jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1370047
Files :
* /giraph/trunk/CHANGELOG
* /giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleTriangleClosingVertex.java
* /giraph/trunk/src/test/java/org/apache/giraph/examples/SimpleTriangleClosingVertexTest.java
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch, GIRAPH-228-2.patch, GIRAPH-228-4.patch, GIRAPH-228-5.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405099#comment-13405099 ]
Eli Reisman commented on GIRAPH-228:
------------------------------------
Hi Avery,
Stupid question (as usual!) but the ArrayListWritable type looks perfect and easy to use too. But, its edu.umb.cloud9 package would need to be added to the maven dependencies for Giraph and it has other things it depends on that now also come along for the ride as I understand it. Is that OK to add if I re-convert my example vertex to use it, or is this why is has not been included so far? Would it be better to suggest its use to multigraph implementors as a self-downloaded jar and forego an example vertex or is it OK to include this dependency change in a patch on JIRA?
If it came down to just pointing people in the right direction, that would be great right there as the first things that come up when trying to figure out how to support arrays is all the hadoop.io stuff that doesn't seem to behave well in Giraph vertices.
The patch I have posted here is to change GIRAPH-217 (SimpleTriangleClosingVertex) into a form that uses Text for final output, and adds a test or two. So its probably a worthwhile improvement over 217 either way. But if I could get the array types into the Giraph POM so others could run the example, I'd be happy to skip this patch and upload another fix using them. I do think there's value in having an array-based example in the examples code.
Thanks for your advice.
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13405165#comment-13405165 ]
Eli Reisman commented on GIRAPH-228:
------------------------------------
ArrayListWritable patch, including change to POM to import edu.umd.cloud9 jar are done and pass mvn verify, but I am not uploading yet. If you guys sign off on adding the library, I can add it. Otherwise the existing patch that outputs text should be put into the repo to fix array probs with GIRAPH-217. Also 217 might have similar problems now with the call to iterator() being removed from the API that my earier 228 patch did.
Thanks again for the advice, I think a multigraph example using ArrayListWritable for <E> param would be in order if this jar is ok to include in the project. Its also easier to work with in compute() than ArrayWritable. Yay!
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch, GIRAPH-228-2.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-228:
-------------------------------
Attachment: GIRAPH-228-2.patch
Needed to fix the iterator() to the new API so the vertex would run. Some changes have happened since i started working on this. Sorry. Still using Text output. Will generate a version using ArrayListWritable but will hold off uploading it until I know if including the library for it in POM is ok, no big deal if not.
Also renamed some unit tests for clarity, passes mvn verify etc.
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch, GIRAPH-228-2.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-228:
-------------------------------
Attachment: GIRAPH-228-5.patch
Had to un-stale this again after the vertex API redesign. Beefed up the tests a little bit, opted for the Guava collections over Java. passes mvn verify etc.
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch, GIRAPH-228-2.patch, GIRAPH-228-4.patch, GIRAPH-228-5.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-228:
-------------------------------
Attachment: GIRAPH-228-4.patch
Sorry for all the confusion. I was able to figure out (duh) how to get the existing ArrayListWritable subclassed in MsgList to work without having to add the cloud9 jar to the POM.xml, and this is the working patch to replace/refactor GIRAPH-217 so it works right with array output, and has new (better) unit tests. Passes mvn verify etc.
Thanks again for your time and advice, Avery!
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch, GIRAPH-228-2.patch, GIRAPH-228-4.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429516#comment-13429516 ]
Jakob Homan commented on GIRAPH-228:
------------------------------------
+1.
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch, GIRAPH-228-2.patch, GIRAPH-228-4.patch, GIRAPH-228-5.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424022#comment-13424022 ]
Eli Reisman commented on GIRAPH-228:
------------------------------------
In case anyone noticed, one of the reasons I keep updating this is that the original SimpleTriangleClosingVertex in GIRAPH-217 that was committed accidentally included a unit test completely commented out. So besides that this is a more robust solution and can act as an example of a tuple Writable in an application (missing from our current example vertices) it also runs all its tests. So thats probably a bonus.
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch, GIRAPH-228-2.patch, GIRAPH-228-4.patch, GIRAPH-228-5.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-228) SimpleTriangleClosingVertex should
not use ArrayWritable for a vertex value
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Reisman updated GIRAPH-228:
-------------------------------
Attachment: GIRAPH-228-1.patch
Changed Vertex Value to Text for storage of final results. Had trouble getting IO formats to work with ArrayWritables of any sort I tried (see separate JIRA) this passes mvn verify and has been run successfully. Adds another unit test to the version in GIRAPH-217.
> SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
> ---------------------------------------------------------------------------
>
> Key: GIRAPH-228
> URL: https://issues.apache.org/jira/browse/GIRAPH-228
> Project: Giraph
> Issue Type: Bug
> Components: examples, test
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Trivial
> Fix For: 0.2.0
>
> Attachments: GIRAPH-228-1.patch
>
>
> Need to convert to (yuck) Text output for vertex values, hope my tuples don't get too long. I followed several instructions from Hadoop about subclassing and using existing ArrayWritables, always both methods will compile with 'mvn verify' nicely, but break in actual use. GiraphJob.run() seems to be the culprit. Don't know if I can solve this myself in a way that will be pleasing to everyone. See GIRAPH-227 if you can help fix this ArrayWritable problem. As far as the vertex example, I'll covert it (and related tests) to Text output now.
> Thanks again to Sean Choi, who noticed this and used the same (Text for tuples) solution successfully. ;)
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira