You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by mangolzy <gi...@git.apache.org> on 2018/03/24 13:40:30 UTC
[GitHub] spark pull request #20899: fix bug in sendMessage() of pregel implementation
GitHub user mangolzy opened a pull request:
https://github.com/apache/spark/pull/20899
fix bug in sendMessage() of pregel implementation
## What changes were proposed in this pull request?
Iterator((edge.dstId, edge.srcAttr._2 * edge.attr))
->
Iterator((edge.dstId, edge.srcAttr._1 * edge.attr))
## How was this patch tested?
Since edge.srcAttr._2 is used to compare with tol, it should be the (newPR - oldPR), but in the sendMessage, the origin code send it as part of the message, instead it should be the newPR which is edge.srcAttr._1.
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mangolzy/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20899.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20899
----
commit be567d83c4069a0592fb61464c795d123bd34116
Author: mangolzy <zi...@...>
Date: 2018-03-24T13:32:52Z
fix bug in sendMessage() of pregel implementation
Iterator((edge.dstId, edge.srcAttr._2 * edge.attr))
->
Iterator((edge.dstId, edge.srcAttr._1 * edge.attr))
Since edge.srcAttr._2 is used to compare with tol, it should be the (newPR - oldPR), but in the sendMessage, the origin code send it as part of the message, instead it should be the newPR which is edge.srcAttr._1.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: Bug fix in sendMessage() of pregel implementation in Pag...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20899
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: Bug fix in sendMessage() of pregel implementation in Pag...
Posted by WenqianZhao <gi...@git.apache.org>.
Github user WenqianZhao commented on the issue:
https://github.com/apache/spark/pull/20899
You can check the [comment](https://github.com/apache/spark/pull/2100#issuecomment-53311125) in pull request [#2100](https://github.com/apache/spark/pull/2100).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: Bug fix in sendMessage() of pregel implementation in Pag...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20899
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: Bug fix in sendMessage() of pregel implementation in Pag...
Posted by WenqianZhao <gi...@git.apache.org>.
Github user WenqianZhao commented on the issue:
https://github.com/apache/spark/pull/20899
> @WenqianZhao I think the point of sending deltas instead of absolute ranks was that, as parts of the graph converge, their deltas would go to zero. GraphX would then be able to compress those zero messages more efficiently.
Got it! Thank you!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: Bug fix in sendMessage() of pregel implementation in Pag...
Posted by WenqianZhao <gi...@git.apache.org>.
Github user WenqianZhao commented on the issue:
https://github.com/apache/spark/pull/20899
By the way, @ankurdave , hi, I'm very curious about why you use delta instead of just newPR as the message. Is that for better performance or just at chance? Could you please answer this question when you are free? Thank you so much.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: fix bug in sendMessage() of pregel implementation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20899
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: Bug fix in sendMessage() of pregel implementation in Pag...
Posted by ankurdave <gi...@git.apache.org>.
Github user ankurdave commented on the issue:
https://github.com/apache/spark/pull/20899
@WenqianZhao I think the point of sending deltas instead of absolute ranks was that, as parts of the graph converge, their deltas would go to zero. GraphX would then be able to stop sending those messages.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20899: Bug fix in sendMessage() of pregel implementation in Pag...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20899
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org