You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Yuanyuan Tian (Created) (JIRA)" <ji...@apache.org> on 2011/12/21 02:19:31 UTC
[jira] [Created] (GIRAPH-112) A bug in
LongDoubleFloatDoubleVertex.write(DataOutput out)
A bug in LongDoubleFloatDoubleVertex.write(DataOutput out)
----------------------------------------------------------
Key: GIRAPH-112
URL: https://issues.apache.org/jira/browse/GIRAPH-112
Project: Giraph
Issue Type: Bug
Components: graph
Affects Versions: 0.70.0
Environment: Any
Reporter: Yuanyuan Tian
Fix For: 0.70.0
I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
for(double msg : messageList.elements()) {
out.writeDouble(msg);
}
Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
The following is a simple solution:
double[] elements=messageList.elements();
for(int i=0; i<messageList.size(); i++) {
out.writeDouble(elements[i]);
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-112) Use elements() properly in
LongDoubleFloatDoubleVertex
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174658#comment-13174658 ]
jiraposter@reviews.apache.org commented on GIRAPH-112:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3287/#review4071
-----------------------------------------------------------
Ship it!
Good catch, this looks great, thanks!
- Jake
On 2011-12-21 07:50:20, Avery Ching wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3287/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-21 07:50:20)
bq.
bq.
bq. Review request for giraph.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. As pointed out by YuanYua, the array returned by elements() cannot have its length used since the array contains all the elements currently stored in the mahout collections, even including invalid elements between size and capacity.
bq.
bq. Whenever possible I converted elements() into forEach(), forEachKey(), forEachPair(). Used size() in other cases.
bq.
bq. Fixed some formatting violations as well in LongDoubleFloatDoubleVertex.java.
bq.
bq.
bq. This addresses bug GIRAPH-112.
bq. https://issues.apache.org/jira/browse/GIRAPH-112
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/LongDoubleFloatDoubleVertex.java 1221634
bq.
bq. Diff: https://reviews.apache.org/r/3287/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Local unittests and MR unittests.
bq.
bq.
bq. Thanks,
bq.
bq. Avery
bq.
bq.
> Use elements() properly in LongDoubleFloatDoubleVertex
> ------------------------------------------------------
>
> Key: GIRAPH-112
> URL: https://issues.apache.org/jira/browse/GIRAPH-112
> Project: Giraph
> Issue Type: Bug
> Components: graph
> Affects Versions: 0.70.0
> Environment: Any
> Reporter: Yuanyuan Tian
> Fix For: 0.70.0
>
> Attachments: GIRAPH-112.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
> for(double msg : messageList.elements()) {
> out.writeDouble(msg);
> }
> Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
> The following is a simple solution:
> double[] elements=messageList.elements();
> for(int i=0; i<messageList.size(); i++) {
> out.writeDouble(elements[i]);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-112) Use elements() properly in
LongDoubleFloatDoubleVertex
Posted by "Avery Ching (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching resolved GIRAPH-112.
--------------------------------
Resolution: Fixed
Assignee: Avery Ching
Thanks for the report and reviews.
> Use elements() properly in LongDoubleFloatDoubleVertex
> ------------------------------------------------------
>
> Key: GIRAPH-112
> URL: https://issues.apache.org/jira/browse/GIRAPH-112
> Project: Giraph
> Issue Type: Bug
> Components: graph
> Affects Versions: 0.70.0
> Environment: Any
> Reporter: Yuanyuan Tian
> Assignee: Avery Ching
> Fix For: 0.70.0
>
> Attachments: GIRAPH-112.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
> for(double msg : messageList.elements()) {
> out.writeDouble(msg);
> }
> Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
> The following is a simple solution:
> double[] elements=messageList.elements();
> for(int i=0; i<messageList.size(); i++) {
> out.writeDouble(elements[i]);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-112) Use elements() properly in
LongDoubleFloatDoubleVertex
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173936#comment-13173936 ]
jiraposter@reviews.apache.org commented on GIRAPH-112:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3287/#review4036
-----------------------------------------------------------
Ship it!
I ran into the same issue yesterday and the solution presented here is correct. For reasons of efficiency, list.elements() returns the internal underlying array for the list, which might be bigger than the number of elements stored in the list. Therefore you should only iterate until list.size() or use the foreachKey() callback.
- Sebastian
On 2011-12-21 07:50:20, Avery Ching wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3287/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-21 07:50:20)
bq.
bq.
bq. Review request for giraph.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. As pointed out by YuanYua, the array returned by elements() cannot have its length used since the array contains all the elements currently stored in the mahout collections, even including invalid elements between size and capacity.
bq.
bq. Whenever possible I converted elements() into forEach(), forEachKey(), forEachPair(). Used size() in other cases.
bq.
bq. Fixed some formatting violations as well in LongDoubleFloatDoubleVertex.java.
bq.
bq.
bq. This addresses bug GIRAPH-112.
bq. https://issues.apache.org/jira/browse/GIRAPH-112
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/LongDoubleFloatDoubleVertex.java 1221634
bq.
bq. Diff: https://reviews.apache.org/r/3287/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Local unittests and MR unittests.
bq.
bq.
bq. Thanks,
bq.
bq. Avery
bq.
bq.
> Use elements() properly in LongDoubleFloatDoubleVertex
> ------------------------------------------------------
>
> Key: GIRAPH-112
> URL: https://issues.apache.org/jira/browse/GIRAPH-112
> Project: Giraph
> Issue Type: Bug
> Components: graph
> Affects Versions: 0.70.0
> Environment: Any
> Reporter: Yuanyuan Tian
> Fix For: 0.70.0
>
> Attachments: GIRAPH-112.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
> for(double msg : messageList.elements()) {
> out.writeDouble(msg);
> }
> Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
> The following is a simple solution:
> double[] elements=messageList.elements();
> for(int i=0; i<messageList.size(); i++) {
> out.writeDouble(elements[i]);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-112) Use elements() properly in
LongDoubleFloatDoubleVertex
Posted by "Avery Ching (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching updated GIRAPH-112:
-------------------------------
Summary: Use elements() properly in LongDoubleFloatDoubleVertex (was: A bug in LongDoubleFloatDoubleVertex.write(DataOutput out))
> Use elements() properly in LongDoubleFloatDoubleVertex
> ------------------------------------------------------
>
> Key: GIRAPH-112
> URL: https://issues.apache.org/jira/browse/GIRAPH-112
> Project: Giraph
> Issue Type: Bug
> Components: graph
> Affects Versions: 0.70.0
> Environment: Any
> Reporter: Yuanyuan Tian
> Fix For: 0.70.0
>
> Attachments: GIRAPH-112.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
> for(double msg : messageList.elements()) {
> out.writeDouble(msg);
> }
> Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
> The following is a simple solution:
> double[] elements=messageList.elements();
> for(int i=0; i<messageList.size(); i++) {
> out.writeDouble(elements[i]);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-112) Use elements() properly in
LongDoubleFloatDoubleVertex
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174686#comment-13174686 ]
Hudson commented on GIRAPH-112:
-------------------------------
Integrated in Giraph-trunk-Commit #58 (See [https://builds.apache.org/job/Giraph-trunk-Commit/58/])
GIRAPH-112: Use elements() properly in LongDoubleFloatDoubleVertex.
(aching)
aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1222071
Files :
* /incubator/giraph/trunk/CHANGELOG
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/LongDoubleFloatDoubleVertex.java
> Use elements() properly in LongDoubleFloatDoubleVertex
> ------------------------------------------------------
>
> Key: GIRAPH-112
> URL: https://issues.apache.org/jira/browse/GIRAPH-112
> Project: Giraph
> Issue Type: Bug
> Components: graph
> Affects Versions: 0.70.0
> Environment: Any
> Reporter: Yuanyuan Tian
> Fix For: 0.70.0
>
> Attachments: GIRAPH-112.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
> for(double msg : messageList.elements()) {
> out.writeDouble(msg);
> }
> Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
> The following is a simple solution:
> double[] elements=messageList.elements();
> for(int i=0; i<messageList.size(); i++) {
> out.writeDouble(elements[i]);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-112) A bug in
LongDoubleFloatDoubleVertex.write(DataOutput out)
Posted by "Avery Ching (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching updated GIRAPH-112:
-------------------------------
Attachment: GIRAPH-112.patch
Actually, elements() is used more than once. This patch should address all of them.
> A bug in LongDoubleFloatDoubleVertex.write(DataOutput out)
> ----------------------------------------------------------
>
> Key: GIRAPH-112
> URL: https://issues.apache.org/jira/browse/GIRAPH-112
> Project: Giraph
> Issue Type: Bug
> Components: graph
> Affects Versions: 0.70.0
> Environment: Any
> Reporter: Yuanyuan Tian
> Fix For: 0.70.0
>
> Attachments: GIRAPH-112.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
> for(double msg : messageList.elements()) {
> out.writeDouble(msg);
> }
> Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
> The following is a simple solution:
> double[] elements=messageList.elements();
> for(int i=0; i<messageList.size(); i++) {
> out.writeDouble(elements[i]);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-112) Use elements() properly in
LongDoubleFloatDoubleVertex
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173932#comment-13173932 ]
jiraposter@reviews.apache.org commented on GIRAPH-112:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3287/
-----------------------------------------------------------
Review request for giraph.
Summary
-------
As pointed out by YuanYua, the array returned by elements() cannot have its length used since the array contains all the elements currently stored in the mahout collections, even including invalid elements between size and capacity.
Whenever possible I converted elements() into forEach(), forEachKey(), forEachPair(). Used size() in other cases.
Fixed some formatting violations as well in LongDoubleFloatDoubleVertex.java.
This addresses bug GIRAPH-112.
https://issues.apache.org/jira/browse/GIRAPH-112
Diffs
-----
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/LongDoubleFloatDoubleVertex.java 1221634
Diff: https://reviews.apache.org/r/3287/diff
Testing
-------
Local unittests and MR unittests.
Thanks,
Avery
> Use elements() properly in LongDoubleFloatDoubleVertex
> ------------------------------------------------------
>
> Key: GIRAPH-112
> URL: https://issues.apache.org/jira/browse/GIRAPH-112
> Project: Giraph
> Issue Type: Bug
> Components: graph
> Affects Versions: 0.70.0
> Environment: Any
> Reporter: Yuanyuan Tian
> Fix For: 0.70.0
>
> Attachments: GIRAPH-112.patch
>
> Original Estimate: 5m
> Remaining Estimate: 5m
>
> I found a bug in LongDoubleFloatDoubleVertex.write(DataOutput out) when running a small graph algorithm. The symptom is that a vertex read from a different worker becomes junk after the RPC communication. And the source of the problem is the writing of the messages in LongDoubleFloatDoubleVertex.write(DataOutput out):
> for(double msg : messageList.elements()) {
> out.writeDouble(msg);
> }
> Here messageList.elements() will returns all the elements currently stored in the mahout DoubleArrayList, even including invalid elements between size and capacity. Therefore, the write() function will write a bunch of invalid messages, which will cause error when reading them back in readfields().
> The following is a simple solution:
> double[] elements=messageList.elements();
> for(int i=0; i<messageList.size(); i++) {
> out.writeDouble(elements[i]);
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira