You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Avery Ching <av...@gmail.com> on 2011/09/16 20:43:44 UTC
Review Request: GIRAPH-34 Failure of Vertex reflection for putVertexList
from GIRAPH-27
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1932/
-----------------------------------------------------------
Review request for giraph.
Summary
-------
The problem shows up when messages are sent locally, and therefore not
reflected. Hence, after the message has been delivered to the other
local vertex, in the next superstep, if the vertex who sent the
message modifies the object, it will modify the message before the
receiving vertex has a chance to access it. Also, combiners maybe
operate on the (shared) message simultaneously with the compute()
method, therefore potential issues could also occur with messages
eventually sent to remote workers. The solution I am proposing is
that all messages should be copied to prevent this error from occuring
as it is pretty tough to debug. An alternative would just be to tell
users to create copies themselves, but it's less convenient and not
intuitive in my opinion.
This addresses bug GIRAPH-34.
https://issues.apache.org/jira /browse/GIRAPH-34
Diffs
-----
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1171389
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/BasicVertex.java 1171389
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/Vertex.java 1171389
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestVertexRangeBalancer.java 1171389
Diff: https://reviews.apache.org/r/1932/diff
Testing
-------
I passed local unittests (mvn package) and unittests on my local
machine's Hadop instance (mvn package
-Dprop.mapred.job.tracker=localhost:50300).
Thanks,
Avery
Re: Review Request: GIRAPH-34 Failure of Vertex reflection for putVertexList
from GIRAPH-27
Posted by Avery Ching <av...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1932/
-----------------------------------------------------------
(Updated 2011-09-16 20:09:21.003725)
Review request for giraph.
Changes
-------
Instead of copying the user messages, simply rely on Javadoc to inform users of the contract with sent message values.
Passed the failing unittest with local Hadoop instance and local unittests.
Summary
-------
The problem shows up when messages are sent locally, and therefore not
reflected. Hence, after the message has been delivered to the other
local vertex, in the next superstep, if the vertex who sent the
message modifies the object, it will modify the message before the
receiving vertex has a chance to access it. Also, combiners maybe
operate on the (shared) message simultaneously with the compute()
method, therefore potential issues could also occur with messages
eventually sent to remote workers. The solution I am proposing is
that all messages should be copied to prevent this error from occuring
as it is pretty tough to debug. An alternative would just be to tell
users to create copies themselves, but it's less convenient and not
intuitive in my opinion.
This addresses bug GIRAPH-34.
https://issues.apache.org/jira /browse/GIRAPH-34
Diffs (updated)
-----
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1171389
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/SimpleCheckpointVertex.java 1171389
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/BasicVertex.java 1171389
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/Vertex.java 1171389
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/test/java/org/apache/giraph/TestVertexRangeBalancer.java 1171389
Diff: https://reviews.apache.org/r/1932/diff
Testing
-------
I passed local unittests (mvn package) and unittests on my local
machine's Hadop instance (mvn package
-Dprop.mapred.job.tracker=localhost:50300).
Thanks,
Avery