You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Alessandro Presta (JIRA)" <ji...@apache.org> on 2013/01/01 21:30:13 UTC

[jira] [Created] (GIRAPH-462) Multithreading breaks out-of-core graph

Alessandro Presta created GIRAPH-462:
----------------------------------------

             Summary: Multithreading breaks out-of-core graph
                 Key: GIRAPH-462
                 URL: https://issues.apache.org/jira/browse/GIRAPH-462
             Project: Giraph
          Issue Type: Bug
            Reporter: Alessandro Presta
            Priority: Critical


~cmartella pointed out this issue: when using multithreaded computation in conjunction with out-of-core graph, we incur in a race condition. The compute threads share the same DiskBackedPartitionStore, whose getPartition() method is not meant to be thread-safe. When two threads request two out-of-core partitions concurrently, they both try to load it to the same slot.
The result is that we can lose the reference to one of the two partitions (which will not be written back to disk) and we can incur in a NullPointerException when both threads are trying to offload the currently loaded partition to disk.

I ran this test to confirm the issue:
https://gist.github.com/4429628
All tests pass except the one that uses both out-of-core graph and multiple compute threads.
The error is the following:
https://gist.github.com/4429650


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira