You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Avery Ching (JIRA)" <ji...@apache.org> on 2012/08/17 10:14:38 UTC
[jira] [Created] (GIRAPH-302) Thread safety issue with sending
partitions around
Avery Ching created GIRAPH-302:
----------------------------------
Summary: Thread safety issue with sending partitions around
Key: GIRAPH-302
URL: https://issues.apache.org/jira/browse/GIRAPH-302
Project: Giraph
Issue Type: Bug
Environment: When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
After digging into the code I found the issue and have fixed it.
Giraph Stats Aggregate edges 99,971,220 0 99,971,220
Superstep 11 0 11
Current workers 300 0 300
Last checkpointed superstep 0 0 0
Current master task partition 0 0 0
Sent messages 0 0 0
Aggregate finished vertices 10,000,000 0 10,000,000
Aggregate vertices 10,000,000 0 10,000,000
This is wrong!
Giraph Stats Aggregate edges 100,000,000 0 100,000,000
Superstep 11 0 11
Last checkpointed superstep 0 0 0
Current workers 300 0 300
Current master task partition 0 0 0
Sent messages 0 0 0
Aggregate finished vertices 10,000,000 0 10,000,000
Aggregate vertices 10,000,000 0 10,000,000
Fixed!
Reporter: Avery Ching
Assignee: Avery Ching
Priority: Critical
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436909#comment-13436909 ]
Avery Ching commented on GIRAPH-302:
------------------------------------
Thanks for the quick review Alessandro!
We can trigger a manual rebuild by logging in to hudson (upper right corner) and then clicking "build now" (which I just did). The error here was a strange ZK reconnect (we have this periodically). Rebuilding should fix it.
12/08/17 10:18:30 INFO zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x139341759650003, likely server has closed socket, closing socket connection and attempting reconnect
12/08/17 10:18:30 WARN zookeeper.ClientCnxn: Session 0x139341759650002 for server janus.apache.org/67.195.138.60:22181, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233)
at sun.nio.ch.IOUtil.read(IOUtil.java:200)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:858)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130)
12/08/17 10:18:30 ERROR zookeeper.ClientCnxn: Error while calling watcher
java.lang.RuntimeException: process: Disconnected from ZooKeeper, cannot recover - WatchedEvent state:Disconnected type:None path:null
at org.apache.giraph.graph.BspService.process(BspService.java:1010)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
12/08/17 10:18:30 FATAL graph.GraphMapper: uncaughtException: OverrideExceptionHandler on thread org.apache.giraph.graph.MasterThread, msg = barrierOnWorkerList: KeeperException - Couldn't get children of /_hadoopBsp/job_local_0008/_applicationAttemptsDir/0/_superstepDir/5/_workerFinishedDir, exiting...
java.lang.IllegalStateException: barrierOnWorkerList: KeeperException - Couldn't get children of /_hadoopBsp/job_local_0008/_applicationAttemptsDir/0/_superstepDir/5/_workerFinishedDir
at org.apache.giraph.graph.BspServiceMaster.barrierOnWorkerList(BspServiceMaster.java:1317)
at org.apache.giraph.graph.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1507)
at org.apache.giraph.graph.MasterThread.run(MasterThread.java:111)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /_hadoopBsp/job_local_0008/_applicationAttemptsDir/0/_superstepDir/5/_workerFinishedDir
at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
at org.apache.giraph.zk.ZooKeeperExt.getChildrenExt(ZooKeeperExt.java:273)
at org.apache.giraph.graph.BspServiceMaster.barrierOnWorkerList(BspServiceMaster.java:1311)
... 2 more
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching updated GIRAPH-302:
-------------------------------
Attachment: GIRAPH-302.patch
Adding patch to match reviewboard https://reviews.apache.org/r/6676/
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching resolved GIRAPH-302.
--------------------------------
Resolution: Fixed
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436919#comment-13436919 ]
Avery Ching commented on GIRAPH-302:
------------------------------------
The next build was successful.
https://builds.apache.org/job/Giraph-trunk-Commit/177/
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Alessandro Presta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436646#comment-13436646 ]
Alessandro Presta commented on GIRAPH-302:
------------------------------------------
Not sure what is happening here. Is there a way we can trigger a rebuild or do we have to wait?
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436645#comment-13436645 ]
Hudson commented on GIRAPH-302:
-------------------------------
Integrated in Giraph-trunk-Commit #176 (See [https://builds.apache.org/job/Giraph-trunk-Commit/176/])
GIRAPH-302: Thread safety issue with sending partitions around. (aching via apresta) (Revision 1374192)
Result = FAILURE
apresta : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1374192
Files :
* /giraph/trunk/CHANGELOG
* /giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Avery Ching updated GIRAPH-302:
-------------------------------
Description:
When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
After digging into the code I found the issue and have fixed it.
Giraph Stats Aggregate edges 99,971,220 0 99,971,220
Superstep 11 0 11
Current workers 300 0 300
Last checkpointed superstep 0 0 0
Current master task partition 0 0 0
Sent messages 0 0 0
Aggregate finished vertices 10,000,000 0 10,000,000
Aggregate vertices 10,000,000 0 10,000,000
This is wrong!
Giraph Stats Aggregate edges 100,000,000 0 100,000,000
Superstep 11 0 11
Last checkpointed superstep 0 0 0
Current workers 300 0 300
Current master task partition 0 0 0
Sent messages 0 0 0
Aggregate finished vertices 10,000,000 0 10,000,000
Aggregate vertices 10,000,000 0 10,000,000
Fixed!
Environment: (was: When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
After digging into the code I found the issue and have fixed it.
Giraph Stats Aggregate edges 99,971,220 0 99,971,220
Superstep 11 0 11
Current workers 300 0 300
Last checkpointed superstep 0 0 0
Current master task partition 0 0 0
Sent messages 0 0 0
Aggregate finished vertices 10,000,000 0 10,000,000
Aggregate vertices 10,000,000 0 10,000,000
This is wrong!
Giraph Stats Aggregate edges 100,000,000 0 100,000,000
Superstep 11 0 11
Last checkpointed superstep 0 0 0
Current workers 300 0 300
Current master task partition 0 0 0
Sent messages 0 0 0
Aggregate finished vertices 10,000,000 0 10,000,000
Aggregate vertices 10,000,000 0 10,000,000
Fixed!)
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Eli Reisman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437135#comment-13437135 ]
Eli Reisman commented on GIRAPH-302:
------------------------------------
Excellent! Right there under our noses the whole time...
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-302) Thread safety issue with sending
partitions around
Posted by "Alessandro Presta (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436644#comment-13436644 ]
Alessandro Presta commented on GIRAPH-302:
------------------------------------------
These concurrency bugs are always nasty. Thanks for fixing!
+1, committed.
> Thread safety issue with sending partitions around
> --------------------------------------------------
>
> Key: GIRAPH-302
> URL: https://issues.apache.org/jira/browse/GIRAPH-302
> Project: Giraph
> Issue Type: Bug
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-302.patch
>
>
> When calling sendPartitionRequest(), we clear the vertex list afterward, making it a race!
> I noticed this when I was running with 300 workers and the number of edges wasn't what I expected. Sometimes we get empty requests!
> After digging into the code I found the issue and have fixed it.
> Giraph Stats Aggregate edges 99,971,220 0 99,971,220
> Superstep 11 0 11
> Current workers 300 0 300
> Last checkpointed superstep 0 0 0
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> This is wrong!
> Giraph Stats Aggregate edges 100,000,000 0 100,000,000
> Superstep 11 0 11
> Last checkpointed superstep 0 0 0
> Current workers 300 0 300
> Current master task partition 0 0 0
> Sent messages 0 0 0
> Aggregate finished vertices 10,000,000 0 10,000,000
> Aggregate vertices 10,000,000 0 10,000,000
> Fixed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira