Re: problem with removeEdges mutation

[José Luis Larroque <la...@gmail.com>]

Hi Michal, i'm facing some strange things with the state of
serializedEdgesByteUsed (i send an email explaining this issue a few days
ago), and maybe both problems are related. Could you find any solution for
this problem?

Thanks in advance!
Jose

2015-02-11 11:43 GMT-03:00 Michal Haris <mi...@visualdna.com>:

> Hi all, I have managed to build a production giraph application off the
> hadoop2 profile (but only mapreduce as yarn has few unresolved bugs). I
> think I have now come across a bug which prevents me from using mutations
> to do  de-duplicate edges in the first superstep of the computation.
>
> Basically whether I use requestRemoveEdges(..) or vertex.removeEdges() the
> program falls over with this exception:
>
> java.lang.IllegalStateException: next: Failed on pos 16 edge
> (targetVertexId = , value = dataxu)
>
> at org.apache.giraph.edge.ByteArrayEdges$ByteArrayEdgeIterator.next(
> ByteArrayEdges.java:170)at org.apache.giraph.edge.ByteArrayEdges.remove(
> ByteArrayEdges.java:111)at
> org.apache.giraph.graph.DefaultVertex.removeEdges(DefaultVertex.java:232)at
> net.imagini.giraph.EdgeDeduplicationJob.compute(
> EdgeDeduplicationJob.java:34)at
> org.apache.giraph.graph.ComputeCallable.computePartition(
> ComputeCallable.java:247)at org.apache.giraph.graph.ComputeCallable.call(
> ComputeCallable.java:168)at org.apache.giraph.graph.ComputeCallable.call(
> ComputeCallable.java:1)at
> org.apache.giraph.utils.LogStacktraceCallable.call(
> LogStacktraceCallable.java:51)at java.util.concurrent.FutureTask.run(
> FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:744)
>
> I have been using my own class for VertexID but even after reverting to
> simple Text to see if problem is with my writable implementation I still
> get what seems a corrupt state of the serialised byte array  edges after
> requesting removal.
>
> There is no mention of this problem in jira and googling yields nothing.
> Anybody tried using removal of edges as part of superstep mutation ?
> Anybody seen something similar?
>
> I am seeing the problem with trunk or release-1.1, haven't tried older.
>
>
>
> --
> Michal Haris
> Technical Architect
> direct line: +44 (0) 207 749 0229
> www.visualdna.com | t: +44 (0) 207 734 7033,
>