You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Vincentius Martin <vi...@gmail.com> on 2014/11/09 07:39:17 UTC

storeCheckpoint() in worker can be slow when a slow worker is presence

Hi all,

I have a question related to my last experience using Giraph.

In Giraph worker's code, I see a line like this:

*"getServerData().getCurrentMessageStore().writePartition(verticesOutputStream,
partition.getId());*"

To the best of my knowledge, while executing this line, a worker writes
some of its partitionMap entry into outputStream. However, with the
existence of a slow workers, this line execution in another worker also
gets slower.

I see that the execution gets faster again after the slow worker has
finished its storeCheckpoint process.

>From what I know, each worker uses its own message store and write to
different output stream. Hence, why the slow storeCheckpoint process in a
worker can affect another worker checkpoint process?

Thanks