Partition Exchange across workers

[Ravikant Dindokar <ra...@gmail.com>]

Hi Giraph users,

I am trying to change the partition to worker assignment, by setting the
Partition Balance Algorithm from static to edges in
balancePartitionsAcrossWorkers method
in org.apache.giraph.partition.PartitionBalancer.java.

I have applied the patch available
https://issues.apache.org/jira/browse/GIRAPH-886

But my application is failing at the end of the first superstep.

Here is what I can see in the Logs :

17/01/02 14:26:35 ERROR yarn.GiraphYarnTask: GiraphYarnTask threw a
top-level exception, failing task
java.lang.RuntimeException: run: Caught an unrecoverable exception null
        at
org.apache.giraph.yarn.GiraphYarnTask.run(GiraphYarnTask.java:104)
        at
org.apache.giraph.yarn.GiraphYarnTask.main(GiraphYarnTask.java:183)
Caused by: java.lang.NullPointerException
*        at
org.apache.giraph.comm.messages.primitives.long_id.LongAbstractMessageStore.getPartitionDestinationVertices(LongAbstractMessageStore.java:123)*
        at
org.apache.giraph.graph.GraphTaskManager.validatePartitioning(GraphTaskManager.java:792)
        at
org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:737)
        at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:337)
        at org.apache.giraph.yarn.GiraphYarnTask.run(GiraphYarnTask.java:92)


Does the BspServiceWorker.exchangeVertexPartitions() method takes care of
moving the messages received by a partition from the previous superstep?

If not, how to move the messages across workers?

Thanks
Ravikant