You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Maja Kabiljo <ma...@fb.com> on 2013/05/06 21:07:48 UTC
Review Request: GIRAPH-656: Input from multiple tables doesn't work with
multithreading
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10958/
-----------------------------------------------------------
Review request for giraph.
Description
-------
When multithreaded input is used, and we try to use MultiEdge/VertexInput, configurations get messed up. This happens because several threads are trying to modify the same Context in a different way. We need to synchronize on context whenever we modify it.
This addresses bug GIRAPH-656.
https://issues.apache.org/jira/browse/GIRAPH-656
Diffs
-----
giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiEdgeInputFormat.java 113b3bc
giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java 631a451
giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeInputFormat.java 928b975
giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java PRE-CREATION
giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexInputFormat.java ed606e3
giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java PRE-CREATION
Diff: https://reviews.apache.org/r/10958/diff/
Testing
-------
Job run with 20 threads
Thanks,
Maja Kabiljo
Re: Review Request: GIRAPH-656: Input from multiple tables doesn't work with
multithreading
Posted by Alessandro Presta <al...@fb.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10958/#review20220
-----------------------------------------------------------
Ship it!
+1
giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java
<https://reviews.apache.org/r/10958/#comment41466>
I guess you meant to add the same comments here.
giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java
<https://reviews.apache.org/r/10958/#comment41467>
...and here.
- Alessandro Presta
On May 6, 2013, 7:07 p.m., Maja Kabiljo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10958/
> -----------------------------------------------------------
>
> (Updated May 6, 2013, 7:07 p.m.)
>
>
> Review request for giraph.
>
>
> Description
> -------
>
> When multithreaded input is used, and we try to use MultiEdge/VertexInput, configurations get messed up. This happens because several threads are trying to modify the same Context in a different way. We need to synchronize on context whenever we modify it.
>
>
> This addresses bug GIRAPH-656.
> https://issues.apache.org/jira/browse/GIRAPH-656
>
>
> Diffs
> -----
>
> giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiEdgeInputFormat.java 113b3bc
> giraph-core/src/main/java/org/apache/giraph/io/formats/multi/MultiVertexInputFormat.java 631a451
> giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeInputFormat.java 928b975
> giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedEdgeReader.java PRE-CREATION
> giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexInputFormat.java ed606e3
> giraph-core/src/main/java/org/apache/giraph/io/internal/WrappedVertexReader.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/10958/diff/
>
>
> Testing
> -------
>
> Job run with 20 threads
>
>
> Thanks,
>
> Maja Kabiljo
>
>