You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Maja Kabiljo <ma...@fb.com> on 2013/05/06 21:07:48 UTC

Review Request: GIRAPH-656: Input from multiple tables doesn't work with multithreading

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10958/
-----------------------------------------------------------

Review request for giraph.


Description
-------

When multithreaded input is used, and we try to use MultiEdge/VertexInput, configurations get messed up. This happens because several threads are trying to modify the same Context in a different way. We need to synchronize on context whenever we modify it.


This addresses bug GIRAPH-656.
    https://issues.apache.org/jira/browse/GIRAPH-656


Diffs
-----

  giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiEdgeInputFormat.java 113b3bc 
  giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java 631a451 
  giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeInputFormat.java 928b975 
  giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java PRE-CREATION 
  giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexInputFormat.java ed606e3 
  giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java PRE-CREATION 

Diff: https://reviews.apache.org/r/10958/diff/


Testing
-------

Job run with 20 threads


Thanks,

Maja Kabiljo


Re: Review Request: GIRAPH-656: Input from multiple tables doesn't work with multithreading

Posted by Alessandro Presta <al...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10958/#review20220
-----------------------------------------------------------

Ship it!


+1


giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java
<https://reviews.apache.org/r/10958/#comment41466>

    I guess you meant to add the same comments here.



giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java
<https://reviews.apache.org/r/10958/#comment41467>

    ...and here.


- Alessandro Presta


On May 6, 2013, 7:07 p.m., Maja Kabiljo wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10958/
> -----------------------------------------------------------
> 
> (Updated May 6, 2013, 7:07 p.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> When multithreaded input is used, and we try to use MultiEdge/VertexInput, configurations get messed up. This happens because several threads are trying to modify the same Context in a different way. We need to synchronize on context whenever we modify it.
> 
> 
> This addresses bug GIRAPH-656.
>     https://issues.apache.org/jira/browse/GIRAPH-656
> 
> 
> Diffs
> -----
> 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiEdgeInputFormat.java 113b3bc 
>   giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java 631a451 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeInputFormat.java 928b975 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java PRE-CREATION 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexInputFormat.java ed606e3 
>   giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/10958/diff/
> 
> 
> Testing
> -------
> 
> Job run with 20 threads
> 
> 
> Thanks,
> 
> Maja Kabiljo
> 
>